Some Thoughts and Questions About the Process


RPG Superstar™ General Discussion

Dedicated Voter Season 7, Star Voter Season 8, Star Voter Season 9

Just some things I've been wondering about.

1. When you see an item often/rarely, does that tell you anything about the item's standing?

2. When you vote tie, does it count as a vote for the purpose of the voter tags and so on?

3. Why when you vote tie does it sometimes show you the same two items again?

4. Although it's cool that some people are so enthusiastic and are voting a thousand times or more, is it possible this allows those people to have too much of a disproportionate effect on the rankings?

5. Should we consider it a problem if we notice the marathon voters' items are particularly well represented in the top 32? Is that mathematically likely to happen given the way the voting is going, and assuming those people are upvoting their items whenever they see them?

6. Should we consider a limit on the number of votes allowed to each person in future years?

By the way, I don't blame anyone for automatically upvoting their own item. I would consider it supererogatory to do otherwise.

Shadow Lodge RPG Superstar 2015 , Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Star Voter Season 9 aka mamaursula

1 person marked this as a favorite.

Given that there are several accounts that an individual voter has seen a single item 5 times (at this time and only to my recollection of having read some of the boards) while they have voted over 1,000 times, you're talking less than 1% of their votes are addressing any one individual item.

Since the rules clearly state that the creator's identity must remain hidden, unless you're suggesting some super secret conspiracy of the Marathon voters to share their information, get other Marathon voters to choose their items and run the risk of being DQ'd by breaking their anonymity to get a few more votes, your concerns are fairly unfounded.

I do believe that those individuals who are most interested and dedicated to this aspect of the game may be likely to be heavily represented in the Top 32 because they are most interested and dedicated to the game as a whole and put forth the same effort in their creation as they do in their voting.

Having read many of the SS threads while voting myself, I find that I have a lot of confidence in the opinions of the Marathon and Dedicated voters and I am quite comfortable that the top 64 items will be the very best items of the submissions.

edited for clarity in first paragraph

Sczarni RPG Superstar 2012 Top 32 , Champion Voter Season 6, Champion Voter Season 7, Champion Voter Season 8, Champion Voter Season 9

1 person marked this as a favorite.
mamaursula wrote:
Given that there are several accounts that an individual voter has seen a single item 5 times maximum (at this time and only to my recollection of having read some of the boards) while they have voted over 1,000 times, you're talking less than 1% of their votes are addressing any one individual item.

In list format (cause lists can be fun occasionally):

  • I have seen my item 7 times.
  • I have seen many items over 20+ times.
  • Today alone, there is one item I have seen 6 times.
  • I have spotted the item leading my list of favorites 16 times.
  • On the opposite end of the spectrum, there are items I rarely see and barely recollect them.

    EDIT: There is an item I have not seen since the cull. Before the cull I saw that item at least 30 times (since I started counting how many times I had seen it). Once, I saw it 3 pairs in a row...

Shadow Lodge RPG Superstar 2015 , Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Star Voter Season 9 aka mamaursula

Thomas LeBlanc wrote:
  • I have seen my item 7 times.
  • I have seen many items over 20+ times.
  • Today alone, there is one item I have seen 6 times.
  • I have spotted the item leading my list of favorites 16 times.
  • On the opposite end of the spectrum, there are items I rarely see and barely recollect them.

    EDIT: There is an item I have not seen since the cull. Before the cull I saw that item at least 30 times (since I started counting how many times I had seen it). Once, I saw it 3 pairs in a row...

Thank you for the update, you overachiever :-)

RPG Superstar 2008 Top 16 , Marathon Voter Season 6 aka Core

The Red Ninja wrote:
1. When you see an item often/rarely, does that tell you anything about the item's standing?

Some have speculated this judging by the patterns. The better items are far less common, and appear to be paired with other good items (though not always). I imagine only the staff knows for sure.

The Red Ninja wrote:
2. When you vote tie, does it count as a vote for the purpose of the voter tags and so on?

As far as I know, yes.

The Red Ninja wrote:
3. Why when you vote tie does it sometimes show you the same two items again?

They tend to re-appear with a different pair from my own observations.

The Red Ninja wrote:
4. Although it's cool that some people are so enthusiastic and are voting a thousand times or more, is it possible this allows those people to have too much of a disproportionate effect on the rankings?

A single voter will have a rather negligible effect compared to the community effort. I've seen my own item 3 times, which is probably less than 1% of the total votes on my item. Probably a lot less than 1%.

The Red Ninja wrote:
5. Should we consider it a problem if we notice the marathon voters' items are particularly well represented in the top 32? Is that mathematically likely to happen given the way the voting is going, and assuming those people are upvoting their items whenever they see them?

As above, I doubt it. The greater community will have the vast majority of the say. From my look at the items it is rather clear what the top 50 or so are. I would be surprised if something other than those 50 make it in the Top 32.

The Red Ninja wrote:
6. Should we consider a limit on the number of votes allowed to each person in future years?

Personally I would say a vote limit per-day. Namely so I'm not tempted to keep alt-tab vote at work :P

RPG Superstar 2014 Top 32 , Marathon Voter Season 6, Dedicated Voter Season 7, Dedicated Voter Season 8, Dedicated Voter Season 9 aka michaeljpatrick

1 person marked this as a favorite.

I expect the Marathon voters to be over-represented in the top 32 because most of us are people who have been following this contest for years and have gone through great effort to understand the design process.

I have only seen my own item twice. Not sure how many times I have voted at this point. More than a thousand, less than five thousand. Probably closer to two thousand.

In any case I don't think that we need to worry about that, because the actual judges will have final say and they aren't going to make decisions based solely on how things are sorted by the voters.

Silver Crusade RPG Superstar 2014 Top 16 , Dedicated Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Marathon Voter Season 9 aka Zahir ibn Mahmoud ibn Jothan

1 person marked this as a favorite.

I'm nowhere near Marathon, but have seen my item 4 times. Some items I have seen a LOT (12+ times).

I saw mine shortly after the cull (and 3 times before), but not again since. Randomness is just so random.

Champion Voter Season 6

Thomas LeBlanc wrote:
mamaursula wrote:
Given that there are several accounts that an individual voter has seen a single item 5 times maximum (at this time and only to my recollection of having read some of the boards) while they have voted over 1,000 times, you're talking less than 1% of their votes are addressing any one individual item.

In list format (cause lists can be fun occasionally):

  • I have seen my item 7 times.
  • I have seen many items over 20+ times.
  • Today alone, there is one item I have seen 6 times.
  • I have spotted the item leading my list of favorites 16 times.
  • On the opposite end of the spectrum, there are items I rarely see and barely recollect them.

    EDIT: There is an item I have not seen since the cull. Before the cull I saw that item at least 30 times (since I started counting how many times I had seen it). Once, I saw it 3 pairs in a row...

Mirrors my voting experience but I am not keeping accurate detailed records. My voting has tapered off but I believe I saw a single "new" item I really liked for the first time on Friday.

Dedicated Voter Season 7, Star Voter Season 8, Star Voter Season 9

1 person marked this as a favorite.
mamaursula wrote:


Since the rules clearly state that the creator's identity must remain hidden, unless you're suggesting some super secret conspiracy of the Marathon voters to share their information, get other Marathon voters to choose their items and run the risk of being DQ'd by breaking their anonymity to get a few more votes, your concerns are fairly unfounded.

I do believe that those individuals who are most interested and dedicated to this aspect of the game may be likely to be heavily represented in the Top 32 because they are most interested and dedicated to the game as a whole and put forth the same effort in their creation as they do in their voting.

edited for clarity in first paragraph

Hmm, I was trying not to overstate the case, but may have done so anyway. My questions weren't statements disguised as questions. I just really don't know the math behind this. If voting a ton of times won't significantly influence it, that's great.

I'm certainly not suggesting any kind of conspiracy. But also, I'm not just talking about people upvoting their own items. Several voters have acknowledged arbitrary prejudices about the way they vote. Indeed, the whole process is arbitrary in many ways. Some people prefer flash over system. Other people are the opposite. Some people just hate certain kinds of items and will always downvote them, etc. So someone voting thousands of times is potentially influencing the process a lot in a way that could be negative.

For example, people talk a lot about how superstar items should be "exciting." What's exciting to me is seeing a well balanced item that I could imagine one of my characters using, not seeing a poorly built, high-pizazz gizmo. I remember in past years there were several entries that made it through by people who had cool ideas but clearly had little to no skill on the technical end. These people invariably were eliminated when they got to the archetype or monster round (where you can't fake or gloss over the system stuff anymore), and it just seemed so predictable and avoidable to me.

My point is actually pretty simple. The judges have said that they didn't expect anywhere near this level of participation, so they probably hadn't considered the potential effects of having people vote 5,000 times, or whatever. You don't need any skills or qualifications to vote 5,000 times except the time and willingness to do it, so if doing it actually allows a mathematically significant influence over the rankings, then there's potential for undesired consequences.

However, it seems that people are telling me that, mathematically, it's not really a concern even if somebody votes 5,000 times. In which case, no problem.

Sczarni RPG Superstar 2014 Top 16 , Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Dedicated Voter Season 9 aka Arkos

2 people marked this as a favorite.

I think that one thing that actually helps this voting method is that we all have different ideas about what it takes to be a superstar. Some want a thorough grasp of system, some want a single great idea, some have a hierarchy... So when it comes down to a final order after tons of votes by people with different priorities, the items at the bottom of the list are the ones that are universally supposed to be at the bottom. Personal taste will lead to some downvotes of a specific item higher on the list (oh no, a misspelling in the first paragraph!), but that will be countered by someone with different priorities who downvotes other items for different reasons.

I have seen my item three times and in each pairing, I felt confident upvoting it. But I just don't have the power to significantly affect my own item!

Of course, a Condorcet system does allow for a vote-ranking strategy, but there's just too many items here to make a real preference list. And if I were worried about that sort of conspiracy, I would worry more about Congressional district gerrymandering than the RPGSS vote-fix!

Marathon Voter Season 6, Marathon Voter Season 7, Star Voter Season 8

1 person marked this as a favorite.
The Red Ninja wrote:


5. Should we consider it a problem if we notice the marathon voters' items are particularly well represented in the top 32? Is that mathematically likely to happen given the way the voting is going, and assuming those people are upvoting their items whenever they see them?

Remember that the Judges still pick the top 32, all our votes do is decide what order the judges look at the items in.... I would be shocked if all the items that make the top 32 are simply the 32 items that get the most votes from the public voting period. Even if John Doe rigged the system somehow to get 1 million up votes against each other item for his misformated Astounding Codpiece of Stupendousness (a Camping Spell in a Can that also makes one immune to a personal range spell when cast by others and has a market price of $4.95), putting it firmly in first place in the voting.. the judges are simply going to reject it and move on.

Marathon Voter Season 6, Marathon Voter Season 7, Star Voter Season 8

1 person marked this as a favorite.

Plus I have been a marathon voter since before they gave us the titles (It is the only voting title I have ever had), and I have only seen my item 4 times... so I doubt that I am unduly influencing its placement.

Dedicated Voter Season 7, Star Voter Season 8, Star Voter Season 9

cwslyclgh wrote:
Plus I have been a marathon voter since before they gave us the titles (It is the only voting title I have ever had), and I have only seen my item 4 times... so I doubt that I am unduly influencing its placement.

Wow, okay. I was under the impression people were seeing their item like once per hundred votes, which made it seem like they could have a statistically significant influence. Anyway, from what people are saying it seems like there's probably no problem.

RPG Superstar 2013 Top 32 , Marathon Voter Season 6, Dedicated Voter Season 7, Dedicated Voter Season 8, Star Voter Season 9 aka Phloid

1 person marked this as a favorite.

I have to be getting pretty close to 2000 votes now and I've only seen my item once and not until I had gone well past marathon status. Sure, I'm jealous of the voters with a similar number of votes who have seen and been able to vote up their own item four or five times, but I'm not really worried that they have gotten an advantage. I think with the judges making the final list of 32, we will get the list as it should be. And there is always a bit of luck with subjective judging. The best you can do is to enter an item that is cool and unique, appeals to the majority of players, and stays away from the judges pet peeves. And then you have to hope that the judges don't like 32 other entries that do these things just a tiny bit more than yours.

RPG Superstar Season 9 Top 32 , Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Marathon Voter Season 9 aka theheadkase

1 person marked this as a favorite.

I have to imagine that since the blog update there are well over 500,000 votes cast. That would mean being a champion voter, if you somehow got the same item every time in your pairings, and you upvoted it each time, would be 1% of total votes. I don't think we have to worry unduly about voters with lots of votes under their belts will affect the outcome in a bad way.

Plus...the judges review the items (and I have a feeling they will be looking at LOTS more than just 32 or 64 from the rankings).

RPG Superstar 2013 Top 32 , Marathon Voter Season 6, Dedicated Voter Season 7, Dedicated Voter Season 8, Star Voter Season 9 aka Phloid

1 person marked this as a favorite.
theheadkase wrote:
Plus...the judges review the items (and I have a feeling they will be looking at LOTS more than just 32 or 64 from the rankings).

I'm not sure you are correct here. i'd be surprised if they look at more than the top 100 and I think they will stop at 64 or maybe even earlier. This system will work better than they expected and they will get the cream rising to the top in such a way that will make the picking of the top 32 pretty easy. And that was the point I guess.

Marathon Voter Season 6, Marathon Voter Season 7, Star Voter Season 8

LOL, I am not sure I share your enthusiasm for the taste of my fellow voters, Phloid.

RPG Superstar Season 9 Top 32 , Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Marathon Voter Season 9 aka theheadkase

1 person marked this as a favorite.
Phloid wrote:
theheadkase wrote:
Plus...the judges review the items (and I have a feeling they will be looking at LOTS more than just 32 or 64 from the rankings).
I'm not sure you are correct here. i'd be surprised if they look at more than the top 100 and I think they will stop at 64 or maybe even earlier. This system will work better than they expected and they will get the cream rising to the top in such a way that will make the picking of the top 32 pretty easy. And that was the point I guess.

Fair enough, although judges are ALSO voting this year, so they've probably seen quite a few of the items already :)

Scarab Sages RPG Superstar 2013 Top 16 , Marathon Voter Season 6 aka Albanus

I think Arkos' point is exactly right. While I personally look more for good comprehension of game mechanics than it seems most people do, I think my votes are only likely to equal out with the people that look more for flash.
And I am now a close to 1000 voter and have seen my own item only twice.

Though regarding the point on spelling: I'm not quite anal enough to have downvoted someone because of a spelling error :P
Unless that error occurred in the title. Then I cried a little and just couldn't help myself.

Shadow Lodge RPG Superstar 2015 , Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Star Voter Season 9 aka mamaursula

The Red Ninja wrote:
Wow, okay. I was under the impression people were seeing their item like once per hundred votes, which made it seem like they could have a statistically significant influence. Anyway, from what people are saying it seems like there's probably no problem.

Not at all. While I am only a Star Voter, I am on the cusp of Dedicated (or at least I believe I am, I'm well over 100) and I have yet to see the one item that I know is in the contest. Which is both kind of disappointing and a relief, because I have not been faced with whether or not I vote for the item I want to go on even if it might not be the stronger of the two items presented. Although given what I've seen so far, I'm pretty sure my decision will be easy should I eventually come to it.

Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Dedicated Voter Season 9

2 people marked this as a favorite.
Adventure Path Charter Subscriber; Pathfinder Rulebook, Starfinder Adventure Path, Starfinder Maps, Starfinder Roleplaying Game Subscriber

The worst possible pairing had just occurred !!!

Mine vs The Wife's!

I know how I want to vote, but I think I will keep my private parts in working order >.<

It is only one vote I suppose :)

Dedicated Voter Season 6

1 person marked this as a favorite.
mamaursula wrote:
Not at all. While I am only a Star Voter, I am on the cusp of Dedicated (or at least I believe I am, I'm well over 100) and I have yet to see the one item that I know is in the contest. Which is both kind of disappointing and a relief, because I have not been faced with whether or not I vote for the item I want to go on even if it might not be the stronger of the two items presented. Although given what I've seen so far, I'm pretty sure my decision will be easy should I eventually come to it.

I feel like you and I are in the same boat. I'm finding myself checking my tag every five or ten votes to see if my tag has rolled over yet. ...I feel like I've voted at least 500 times by now, but apparently not. My perception may be horribly skewed, though. ...But yeah, I haven't seen my item, either.

Probability is cruel. The chances of seeing any given item multiple times is pretty good, but the chances of seeing any specific item are comparatively low.

From what I'm seeing and reading, though, I really do think things will balance out overall. There are plenty of studies about how the hive mind is more accurate than any individual (see here) Meaning, that if you ask a group of 100 people to estimate how many jelly beans are in a jar, the average of all of the answers is likely to be as accurate as the most accurate indivudal answer (if not more so).

Not exactly the same thing, but I think the same general idea applies.

Shadow Lodge RPG Superstar 2015 , Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Star Voter Season 9 aka mamaursula

3 people marked this as a favorite.
Anthony Adam wrote:

The worst possible pairing had just occurred !!!

Mine vs The Wife's!

I know how I want to vote, but I think I will keep my private parts in working order >.<

It is only one vote I suppose :)

That's what the "Both items are equally good" button is for Anthony!

Marathon Voter Season 6, Marathon Voter Season 7, Star Voter Season 8

1 person marked this as a favorite.

Possibly the only time it is the right button to use.

Sczarni RPG Superstar 2014 Top 16 , Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Dedicated Voter Season 9 aka Arkos

Albanus wrote:

Though regarding the point on spelling: I'm not quite anal enough to have downvoted someone because of a spelling error :P

Unless that error occurred in the title. Then I cried a little and just couldn't help myself.

See? Then our votes will support the system. If we've both downvoted something, it will be because it failed both of our requirements. If we both upvoted an item, it will be because both of us were satisfied, even though we looked for different things. The items where we disagree get stuck in the middle, and another voter will have to sort out which of our requirements they tend to agree with. It's all for the greater good!

And, as the unfortunate teacher of a yearly SAT prep program, I am a little conscious of spelling errors. It certainly isn't an instant downvote for me, but it will absolutely break ties.

RPG Superstar 2012 Top 16 , Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Dedicated Voter Season 9 aka motteditor

Anthony Adam wrote:

The worst possible pairing had just occurred !!!

Mine vs The Wife's!

I know how I want to vote, but I think I will keep my private parts in working order >.<

It is only one vote I suppose :)

It's anonymous, Anthony. She'll never know if you don't tell her. : )

RPG Superstar 2010 Top 32, 2011 Top 4 , Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Marathon Voter Season 9 aka DankeSean

Jacob W. Michaels wrote:
Anthony Adam wrote:

The worst possible pairing had just occurred !!!

Mine vs The Wife's!

I know how I want to vote, but I think I will keep my private parts in working order >.<

It is only one vote I suppose :)

It's anonymous, Anthony. She'll never know if you don't tell her. : )

Unfortunately, a lot of spouses come with permanent zone of truth auras attached.

Marathon Voter Season 6

Yeah. I reached marathon and am aiming for Champion but I still haven't seen my item.

Paizo Employee Chief Technical Officer

Anthony Adam wrote:

The worst possible pairing had just occurred !!!

Mine vs The Wife's!

I know how I want to vote, but I think I will keep my private parts in working order >.<

It is only one vote I suppose :)

You asked for it, Mr. Marathon....

Shadow Lodge RPG Superstar 2015 , Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Star Voter Season 9 aka mamaursula

Vic Wertz wrote:
Anthony Adam wrote:

The worst possible pairing had just occurred !!!

Mine vs The Wife's!

I know how I want to vote, but I think I will keep my private parts in working order >.<

It is only one vote I suppose :)

You asked for it, Mr. Marathon....

Wait, is that all I had to do was ask!?

RPG Superstar 2015 Top 8 , Star Voter Season 6, Star Voter Season 7, Star Voter Season 8 aka DeathQuaker

mamaursula wrote:
Vic Wertz wrote:
Anthony Adam wrote:

The worst possible pairing had just occurred !!!

Mine vs The Wife's!

I know how I want to vote, but I think I will keep my private parts in working order >.<

It is only one vote I suppose :)

You asked for it, Mr. Marathon....
Wait, is that all I had to do was ask!?

I think what he's saying is that since Anthony Adam is a marathon voter, the odds of that happening became more likely.

Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Dedicated Voter Season 9

Adventure Path Charter Subscriber; Pathfinder Rulebook, Starfinder Adventure Path, Starfinder Maps, Starfinder Roleplaying Game Subscriber

Lol, I certainly didn't ask for it :P

And with the vote I made, I didn't get a "You're asking for is, mister!" from her either :P

In fact, I think she made me a steaming cup of dark cherry swiss chocolate. And very nice it was too!

Grand Lodge RPG Superstar 2012 Top 32 , Marathon Voter Season 6, Dedicated Voter Season 7, Dedicated Voter Season 8 aka Jiggy

1 person marked this as a favorite.
Anthony Adam wrote:
I think she made me a steaming cup of dark cherry swiss chocolate.

You're not sure?

Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Dedicated Voter Season 9

Adventure Path Charter Subscriber; Pathfinder Rulebook, Starfinder Adventure Path, Starfinder Maps, Starfinder Roleplaying Game Subscriber

It was soooo nice, I got choccy high. What can I say?

Owner - House of Books and Games LLC , Marathon Voter Season 6, Star Voter Season 7

1 person marked this as a favorite.

I'd say "say nothing." You won't necessarily succeed, but you're less likely to fail :)

Community / Forums / Archive / Paizo / RPG Superstar™ / General Discussion / Some Thoughts and Questions About the Process All Messageboards

Want to post a reply? Sign in.
Recent threads in General Discussion