I'm pretty sure that the voting system is supposed to show 2 items that you have not seen before. This is not the case. Just letting you know.
The system is designed so that items that have been viewed the least may show up slightly more frequently. You may see the same item more than once as only a few people have viewed it.
This way it makes sure all items get the same coverage.
Exactly. You'll never see the same pairing, but you may very well see the same items more than once.
It will also give an important link to the chain of logic at play...as the voting isn't really just tallying votes.
Contributor
,
Dedicated Voter Season 6, Dedicated Voter Season 7, Dedicated Voter Season 8, Star Voter Season 9
aka Steven T. Helt
|
| 2 people marked this as a favorite. |
In the first dozen pairings I saw tonight, I saw one item three times. I'm concerned because most of the items I've seen are not keepers, but this item, kind of meh, got three votes because it was stacked up against terrible items.
I recognize no system is perfect, and it looks like the frequency of this item might be balanced out based on no one else seeing it. But if there are hundreds of items, three votes in ten minutes seems a decent head start.
In the first dozen pairings I saw tonight, I saw one item three times. I'm concerned because most of the items I've seen are not keepers, but this item, kind of meh, got three votes because it was stacked up against terrible items.
I recognize no system is perfect, and it looks like the frequency of this item might be balanced out based on no one else seeing it. But if there are hundreds of items, three votes in ten minutes seems a decent head start.
Keep in mind that we're not asking which of these items should make the op 32, we're asking which of the two is better. Being better than three bad items just tells us it's better than those three items, not that it got three generic votes.
|
Contributor
,
Dedicated Voter Season 6, Dedicated Voter Season 7, Dedicated Voter Season 8, Star Voter Season 9
aka Steven T. Helt
|
| 2 people marked this as a favorite. |
Sure...I trust the system...just feels weird to vote for the same bad item three times.
I obey. Hurts to cast a vote sometimes, though. :b
I have seen the same pairing twice. In each instance I chose to not vote for either as both items were in a way...terrible? I feel for the Judges as some of these items are just....wow.
Ties get shuffled back into the deck, so it is unlikely but not impossible for the same person to see them again.
| 1 person marked this as a favorite. |
At this point I am seeing one repeat item about 1/4 to 1/3 of the time, and I even saw one that was a double repeat (as in both items were repeats, but not matched against the same item as before)... and I have twice got the same item two times in a row.
| 1 person marked this as a favorite. |
I've never seen the exact same pairing but I've seen items up to 5 times already - also with item A is a 3 timer and item B is a 2 timer
I am trusting the math and the judges making the final call.
- I think we have to keep in mind we are just trying to weed out the wheat from the chaff so the judges only have have to view maybe the top 100 or 1000 items rather than 10000 or 50000 -with 40000 of them being terrible, 5000 bad, 4000 decent but not superstar and the final group has potential and let the pros make the final call to pare it down to 32.
after looking at 20-30 pairings I can't image how grueling it would be to try to look at 5000 or more
on the plus side, seeing all the terrible ones makes me feel better about my shot and makes me wonder how so many people didn't read the rules.
not knowing your item was basically already created in some lesser known peripheral work is easy to understand, but messing up the formatting is like misspelling your name at the top of the test - it's a gimme
also, not a rule, but so much of the ".....this was created by the high muckety-muck for raking muck in his deep dark muck pit and is now favoured by the mudmen of muck town for its super muck raking abilities"
- the judges have been screaming to leave that kind of text out for years now and now I know why - it is hard to read the item and it makes it so hard to see the possible gem in all the ... mucky, pointless, text.
if your item is good, we will be able to figure out who will get the most out of it, seriously.
Exactly. You'll never see the same pairing, but you may very well see the same items more than once.Not so. I got the same pairing one right after the other. The only difference is their placement was reversed.
Oooooh. That sounds like a bug, unless the first time you chose "vote for neither". Plus, statistically impressive coincidence! Ross?
Exactly. You'll never see the same pairing, but you may very well see the same items more than once.Not so. I got the same pairing one right after the other. The only difference is their placement was reversed.
Same thing happened to me.
Ties get shuffled back into the deck, so it is unlikely but not impossible for the same person to see them again.
It happen to me once I passed on voting on pair items only to have them both immediately reappear, only difference is that the items switch sides.
I just saw the two same items against each other, back to back. The only difference was that they changed place. The one on the right was on the left and the one on the left was on the right. This happened right after I voted for neither of them. I wonder if that has an impact.
| 1 person marked this as a favorite. |
Actually saw the exact same pairing back-to-back, except they swapped positions.
| 1 person marked this as a favorite. |
I just saw the two same items against each other, back to back. The only difference was that they changed place. The one on the right was on the left and the one on the left was on the right. This happened right after I voted for neither of them. I wonder if that has an impact.
That's pretty much what I was expecting. People seem to think the 'vote for neither' really means 'vote both down'. It doesn't. It means "someone else will vote on this pairing later."
What Ronars said. Picking "neither" just throws the pair back into the "items not viewed yet this cycle" pool, and if the pool is small, the chance of getting those two items back is pretty good.
Exactly. And while it won't happen every time, each time the cycle goes through each item to be shown, the chances of hitting neither and then getting one or both of the same items again rises.
| 3 people marked this as a favorite. |
i think that 'back in the deck' mechanic might be part of the reason that we're seeing so many bad items... i sometimes hit 'neither' just because i can't bring myself to actually cast a vote for either item. its a shame that there can't be a "please vote against both of these" button.
also, i've voted enough times that i'm getting a repeat item 2/3 or 3/4 of the time (and occasionally pairs where both are repeats). i know that statistically i probably shouldn't be too worried that i haven't seen my own item yet, but i'm terrified that something happened and i got left out of the pool (or, worse, DQ'ed). anyone else having a similar experience?
edit: i've decided that not having seen my own item is good- it means it hasn't been getting kicked back into the queue by people who can't bring themselves to vote for either item... (at least, that's what i'm going to keep telling myself)
edit: i've decided that not having seen my own item is good- it means it hasn't been getting kicked back into the queue by people who can't bring themselves to vote for either item... (at least, that's what i'm going to keep telling myself)
Me, too.
| 1 person marked this as a favorite. |
I'm also seeing the same item regularly, even though I've voted against it three times now.
I've seen several repeats, as well. Reading this thread actually makes me a little more confident in voting when I don't like either item, as I now have a little more understanding of and faith in the sorting/voting process. I imagine we'll be voting on pairings of increasingly similar viability as the voting period wears on.
You aren't voting for 'this item should advance/that item should not advance'.
You are voting for 'this item should be judged before that item.'
The final result isn't a score, just a ranking.
| 1 person marked this as a favorite. |
...You are voting for 'this item should be judged before that item.'
The final result isn't a score, just a ranking.
That's the best phrasing of the vote description I've heard so far.
"look here first"
RPG Superstar 2012 Top 32
,
Dedicated Voter Season 6, Marathon Voter Season 7, Dedicated Voter Season 8, Star Voter Season 9
|
I believe the reason for seeing the same items again and again is that a lot of people rejected to vote for or against them. The only good reason not to vote is that the two items are so evenly matched that it's hard to decide. I guess this occurs more often when you have two pick one of two bad items, and it means that the items that you see a lot are at the bottom of the pile, so to say. If you don't see your item at all despite voting dozens upon dozens of times, your item is probably popular.
My last 10 pairings had only two or three new items, all the rest were repeats, and I have barely used the "neither" button.
| 1 person marked this as a favorite. |
I've seen the same item show up four times now. I've voted for it once based on the pairing and gone the other way three times.
This voting system is trying way too hard to sell me on this particular item. It doesn't even come with cup holders!
| 2 people marked this as a favorite. |
I've seen the same item show up four times now. I've voted for it once based on the pairing and gone the other way three times.
This voting system is trying way too hard to sell me on this particular item. It doesn't even come with cup holders!
Alternatively, it casts feather fall, fireball, acts as a Rod of Wonder on alternating Tuesdays, has a table for the random effect it generates on a full moon, is prized by dwarves, and comes with cup holders.
I believe the reason for seeing the same items again and again is that a lot of people rejected to vote for or against them. The only good reason not to vote is that the two items are so evenly matched that it's hard to decide. I guess this occurs more often when you have two pick one of two bad items, and it means that the items that you see a lot are at the bottom of the pile, so to say. If you don't see your item at all despite voting dozens upon dozens of times, your item is probably popular.
How people vote does not impact the order they are presented (other than ties being thrown back into the pool).
Alternatively, it casts feather fall, fireball, acts as a Rod of Wonder on alternating Tuesdays, has a table for the random effect it generates on a full moon, is prized by dwarves, and comes with cup holders.I've seen the same item show up four times now. I've voted for it once based on the pairing and gone the other way three times.
This voting system is trying way too hard to sell me on this particular item. It doesn't even come with cup holders!
But does it come in red? I heard the red ones cast faster.
|
RPG Superstar 2012 Top 32
,
Dedicated Voter Season 6, Marathon Voter Season 7, Dedicated Voter Season 8, Star Voter Season 9
|
I believe the reason for seeing the same items again and again is that a lot of people rejected to vote for or against them. The only good reason not to vote is that the two items are so evenly matched that it's hard to decide. I guess this occurs more often when you have two pick one of two bad items, and it means that the items that you see a lot are at the bottom of the pile, so to say. If you don't see your item at all despite voting dozens upon dozens of times, your item is probably popular.How people vote does not impact the order they are presented (other than ties being thrown back into the pool).
If I recall correctly, someone from Paizo said that items that have received fewer votes are more likely to be displayed (and with that I mean the total number of votes, for or against). And if ties are shuffled back to the deck (i.e. neither item got a vote) it means that items that receive more "neither" votes are more likely to be displayed, correct?
Ties between bad vs bad, mediocre vs mediocre and good vs good items are all possible, but I presume good items are less common than bad ones, so ties between two bad items occur more often than ties between two good items. Which in turn means, bad items are shuffled back to the deck more often than good items, and consequently, this means that items that are frequently displayed are generally less popular ones.
However, if my assumptions are incorrect (i.e. the total number of votes does not affect the chance of an item being displayed or ties do count when determining the total number of votes or people really do hit "neither" just as often when they see items they think are good), then my theory totally fails. :)
And it should be noted that not all of the frequently repeating items have been super bad.
Dedicated Voter Season 6, Star Voter Season 7
|
Exactly. You'll never see the same pairing, but you may very well see the same items more than once.
Not true ... I have gotten the same pairing (though their positions were reversed).
EDIT: and this was already raised ... that'll teach me to reply before reading through a thread ... no, probably not. ;)
For some reason the first day I voted all the items were bad to mediocre, but today all the items are steadily seeming better and better. It makes me both relieved (that there are quality items out there) and worried (that I have quality competition ;p).
I think it's the rewording of the buttons that has really helped. I'm starting to see new items almost every time now.
Not that I think this is a good idea, exactly, but another way the system could be made to work is to have people choose the less superstar-worthy item of the two. People seem to have an easier time saying "this item is clearly bad" than "this item is clearly vote-worthy". Something like:
BLAM #1
BLAM #2
BLAM BOTH
The problem is that it would set the tone of the competition to be really negative.
|
RPG Superstar 2012 Top 32
,
Dedicated Voter Season 6, Marathon Voter Season 7, Dedicated Voter Season 8, Star Voter Season 9
|
Or instead of shuffling ties back to the deck, both get just 1/2 vote (as opposed to 1 vote for the winner and 0 for the loser). That way, the overall number of votes is the same regardless of which button you click, and you don't have to view the same items over and over again.
Or instead of shuffling ties back to the deck, both get just 1/2 vote (as opposed to 1 vote for the winner and 0 for the loser). That way, the overall number of votes is the same regardless of which button you click, and you don't have to view the same items over and over again.
They aren't votes. They are comparative rankings.
I suppose it could link them together; if you vote a tie for a pairing, it doesn't rank them relative to each other, but every time you rank something else as above or below one of those two, you also rank it above or below the other. I don't believe that is how it is implemented here.
| 2 people marked this as a favorite. |
Or instead of shuffling ties back to the deck, both get just 1/2 vote (as opposed to 1 vote for the winner and 0 for the loser). That way, the overall number of votes is the same regardless of which button you click, and you don't have to view the same items over and over again.
That isn't how Condorcet voting works though. Condorcet voting creates a set of A>B values and uses those values as its mathematical input. Though it seems like it would be easy to also apply A=B, the system doesn't work that way. Voting for Item A does not give it a +1, and voting against Item A does not give it a -1. It only establishes a relationship of A>B or B>A. Voting neither is like saying "These are too close to tell. It is beyond me to compare them, and I need someone else to do it."
To be entirely honest, I think that including the tie button has only led to confusion and less optimized sorting.
Keep in mind that if an item loses in future pairings, that means that every item it beat gets bumped down the list with it. And if an item wins in future pairings, every item it lost to gets bumped up on the list with it. If we were voting initially on items that had made a cut and were all good, then this would become gritty fast, and one bad vote could make things ugly. But we have enough filler and junk items in there, that any links we form are only going to help pull the great things to the top.
You aren't voting for 'this item should advance/that item should not advance'.
You are voting for 'this item should be judged before that item.'
The final result isn't a score, just a ranking.
Therein lies the flaw....if item x is horrible but slightly better than y...and get seen 150 times more than item z...which is amazing....will item x make it to top 64...while item z never gets seen by the actual judges....?
|
RPG Superstar 2012 Top 32
,
Dedicated Voter Season 6, Marathon Voter Season 7, Dedicated Voter Season 8, Star Voter Season 9
|
Well yes, I understand the principle of linking/chaining, but I don't know how it works mathematically. At any rate, the tie button does make things more confusing, and it's annoying to see the same items over and over again, whatever the reason.
Therein lies the flaw....if item x is horrible but slightly better than y...and get seen 150 times more than item z...which is amazing....will item x make it to top 64...while item z never gets seen by the actual judges....?You aren't voting for 'this item should advance/that item should not advance'.
You are voting for 'this item should be judged before that item.'
The final result isn't a score, just a ranking.
So what happens with rock-paper-scissors?
I really wish there was an option for "both are great" and another for "both are terrible." Every person should have their own lists, so the same combination could appear to 100 people. If I am a terrible judge of items, does someone else get to also vote on whether "paper" really does beat to "scissors"?
What Ronars said. Picking "neither" just throws the pair back into the "items not viewed yet this cycle" pool, and if the pool is small, the chance of getting those two items back is pretty good.
I've been seeing a lot of item pairs like this, where both items are equally bad, ok, or good.
It's a hard choice to make and it takes time, but someone has to do it.
I wish there was a way to vote both items down or up.
Also, I wish we didn't have to see obviously bad items after a few days. In a way the judges had it easier in the sense that it would only take one judge to reject an entry and then the other judges would never have to view it. I've seen the same item 5+ times that has no chance of making it into the top 32. Is there really a purpose to sorting through the bottom 20% (unless they appeared there by accident)?
Star Voter Season 6, Star Voter Season 7
|
I see repeated items every 2-3 pairings, often with the exact same pairing as it was last time.
I didn't vote for them before because, well, they're both bad. I really wish we could have a button that basically says 'Please, never show me these items again.' I won't vote for an item that's bad simply because it's "less bad" than the other one. :/
|
RPG Superstar 2012 Top 32
,
Marathon Voter Season 6, Dedicated Voter Season 7, Dedicated Voter Season 8
aka Jiggy
|
I won't vote for an item that's bad simply because it's "less bad" than the other one. :/
Spoken like someone who hasn't paid a lick of attention to any of the people who wrote the code describing exactly what your "votes" do and don't mean.
|
Star Voter Season 6, Star Voter Season 7
|
Unfortunately, Jacob, the trouble with voting for an item in that particular case still grants weight to the individual item. Given enough instances in the dataset, the variance can grow wildly out of control.
I would still rather see a weight of 0 when both items are designed outside of the scope of the event, rather than an aritifically-inflated weight granted by an ill-informed vote.
I am pretty sure it doesn't work that way... voting doesn't 'grant weight' to an item at all, all it does is ranks it against the specific other item presented, the program that they are using then compares these relations ships between every other item... so if a lot of people vote crap item one better then crap item 2, it will rank higher than an item that people consistently vote worse than crap item 2, but not higher than one that people consistently vote better than crap item 1 (or better than an item voted better than crap item one), no matter how few votes (via strange probability variation say) that the better item might get.
Or as Vic said:
"downgrading" both is not part of the system. You're not providing cumulative "pro" or "anti" votes with this system; you are simply providing an opinion of one item relative to the item it is displayed against, with no bearing to other items. At the end, a lot of math is done to create a full ranking.