If voting continues to next year may I suggest a "Neither of the two" and "reshuffle" buttons instead of the current "neither"
"Neither of the two" would downgrade both
and
"reshuffle" would queue the items for comparison against other items
Chief Technical Officer
|
| 1 person marked this as a favorite. |
"downgrading" both is not part of the system. You're not providing cumulative "pro" or "anti" votes with this system; you are simply providing an opinion of one item relative to the item it is displayed against, with no bearing to other items. At the end, a lot of math is done to create a full ranking.
I assume your using some variant of a chess player ranking system
maybe all items are treated as winning 100 (or 1k) matches before the voting starts, and the downgrade treats as a weak loss for both items. At 0 points and 0 wins the items is no longer displayed for ranking
Is it possible for a non-disqualified item to be removed from ranking? Like if an item gets X amount of up votes or X amount of down votes does it go directly into a keep or trash pile? Or would any voter eventually get to see every item provided they keep voting long enough?
| 2 people marked this as a favorite. |
Is it possible for a non-disqualified item to be removed from ranking? Like if an item gets X amount of up votes or X amount of down votes does it go directly into a keep or trash pile? Or would any voter eventually get to see every item provided they keep voting long enough?
The question is not how many positive/negative votes received, but against whom.
I think the bottom 10-20%, after a few days' voting, could be easily discarded. But if you get two items in the bottom 10%, one of them will get an upvote. And even a fabulously awesome item can get downvoted if crossed with an even better item.
I'd love to see a gradual trimming of the item pool, because I think a ton of our effort is going to go into establishing a precise rating for a bunch of items that would be rejected on sight in previous years, and will still absolutely wind up at the low end of the ranking this year. It doesn't sound like that's being planned, though, at least this year.
I had the same thought Standback, with the judges being the final arbiters. After a week, judges look at the bottom X% and vote reject (or not). After the next week do it again.
But then it does not help the judges lesson their load which is a major part of the crowd sourcing. And that probably does not even take into account mathmagical numbers-changes that would create for Vic, Ross, Gary, & company. Still this could be a suggestion for future years :)
| 1 person marked this as a favorite. |
We've been running some numbers behind the scenes. Not ready to talk about it yet, but the system as set up is doing a good job of pushing the worst items to the bottom and the better items to the top.
I don't know if we'll do a purge of those bottom items in early January (most of Paizo is out this coming week), or if such at thing will even be necessary (Gary and Ross know the math of it better than I do, the system may sort itself out once it has a certain number of downranked items).
I would say "sure, trim the items that couldn't possibly win," but the down side is for people who are determined to find out if their item was disqualified - if some of the items are droppped off the list, there's literally no way for them to find out.
As opposed to now, where it's merely very, very difficult for someone to determine if their item is still in the running.
I assume your using some variant of a chess player ranking system
Nope. They're doing Schulze (a system for preferential elections invented in 1997), which is very, very different from Elo rankings.
RPG Superstar 2012 Top 32
,
Champion Voter Season 6, Champion Voter Season 7, Champion Voter Season 8, Champion Voter Season 9
|
We've been running some numbers behind the scenes. Not ready to talk about it yet, but the system as set up is doing a good job of pushing the worst items to the bottom and the better items to the top.
I don't know if we'll do a purge of those bottom items in early January (most of Paizo is out this coming week), or if such at thing will even be necessary (Gary and Ross know the math of it better than I do, the system may sort itself out once it has a certain number of downranked items).
By the end of the weekend the bottom 10% should be safe to drop. And another 10% at the end of the next week.
I feel as though yesterday I spent 3 hours sorting the bottom of the pile of items people voted "Both items are equally good/bad" or "Vote for Neither".