A New-Found Respect For The Judges


RPG Superstar™ General Discussion

Dark Archive Dedicated Voter Season 6, Star Voter Season 7

3 people marked this as a favorite.

After looking at about 20 pairs of items and clicking "vote for neither" I can understand the general grumpiness of the judges and why Paizo went with this voting system--it is the save what little sanity they have left.
I wanted to poke my eyes out after only 20 pairs of items. I can't imagine spending hours going through all of these.

Bless you, you are better than I.

and yes, I did submit an item for this year's contest

Scarab Sages Marathon Voter Season 7

There are some very interesting ideas I have seen so far, but most of my pairings are getting the "neither" vote as well. I haven't seen more than a handful of what i would call bad ones, but the majority are nevertheless not quite as polished as I think a Superstar item should be.

Marathon Voter Season 6

1 person marked this as a favorite.

I was actually surprised by how not-bad many of these were. I was expecting fire and brimstone, and am pleasantly surprised at the quality.

I've come to accept the fact that I'll need to look past item formatting for minor things though.

RPG Superstar 2013 Top 16 , Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Dedicated Voter Season 9 aka Darkjoy

I've seen some good items, at least compared to the other one under review. I did push neither a few times.

I knew I would be exposed to some bad entries but some are really bad.

RPG Superstar 2012 Top 16 , Marathon Voter Season 6, Marathon Voter Season 7, Dedicated Voter Season 8

I think the public vote will be a better educational tool into what makes a good item than most of the other advice out there. You get good at spotting obvious mistakes REAL quick.


I was just going to start his thread. My respect for the judges from the prior years has increased a thousand-fold.

I will say, I don't think the neither option is meant for disdain; but, as a measure to eliminate ties. The items will show up again as you continue with your voting. I think it has been programmed as a reshuffle: not a judgement.

Star Voter Season 6

I've gone through a dozen at this point - I voted neither for six of those pairs. Three others were kind of interesting, better than the other at least, so I voted for them. The other three I voted for weren't as polished as I would like, but had some cool ideas, so I saved them.

Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Dedicated Voter Season 9

Adventure Path Charter Subscriber; Pathfinder Rulebook Subscriber

Agreed. The judges are stellar, and now we know why.

Eye Bleed items - yeah, seen some...
Fat Template Fu - yeah, he will need kids to help eat the template errors
Cool items - seen a couple
Is it fun - you better believe it... so much so that

If you ever conceive taking my vote button away, there will be ramifications... like no Christmas for you!

RPG Superstar 2013 Top 16 , Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Dedicated Voter Season 9 aka Darkjoy

Wild Gazebo wrote:

I was just going to start his thread. My respect for the judges from the prior years has increased a thousand-fold.

I will say, I don't think the neither option is meant for disdain; but, as a measure to eliminate ties. The items will show up again as you continue with your voting. I think it has been programmed as a reshuffle: not a judgement.

Neither for me at least means that both are bad and undeserving of my vote.

Paizo Employee Chief Technical Officer

1 person marked this as a favorite.

Clicking "neither" on everything is actually not as helpful as choosing the one of them that's better, even if "better" is still "not good". Please only use "neither" when you really can't choose between them.


Good to know. I used neither a lot.

Shadow Lodge Star Voter Season 6, Star Voter Season 7, Star Voter Season 9

well .. i was feeling bad about constantly choosing neither.
I was asking myself if I was too much critic just because I'm in the competition... but apparently not... many items are just bad....

And a lot of item openly disregards any possible suggestion given to the contestants...
I have already read at least 5 different pieces of technology disguised (and badly) as magic.

Designer, RPG Superstar Judge

Just remember, there are HUNDREDS AND HUNDREDS of items submitted each year, and only THIRTY-TWO make it to the next round. That means hundreds are rejected for various reasons. And even if only 25% are rejected for being really bad, that's still HUNDREDS of items that are... destined for infamy, so to speak. :)


Spell-in-a-can! Spell-in-a-can!

Is there a chance that the next year we could get ability to comment on each item so their authors could see the critique after the voting ends?

Paizo Employee PostMonster General

2 people marked this as a favorite.

We've clarified the FAQ for when you should select "neither" as an option.

Do I have to express a preference?
No, but the ranking method we're using works much better if you do. If you think two items are both bad, pick the one that is less bad. If they're both awesome, pick the one that is more awesome. If you think the two items really are exactly as good or as bad as each other, click "Vote for Neither."

In other words, "neither" should be a last, not first, choice.

Shadow Lodge Star Voter Season 6, Star Voter Season 7, Star Voter Season 9

Gary Teter wrote:

We've clarified the FAQ for when you should select "neither" as an option.

Do I have to express a preference?
No, but the ranking method we're using works much better if you do. If you think two items are both bad, pick the one that is less bad. If they're both awesome, pick the one that is more awesome. If you think the two items really are exactly as good or as bad as each other, click "Vote for Neither."

In other words, "neither" should be a last, not first, choice.

Sure, I read it ... And I'm doing it now...

I just still feel a little dirty inside...

Marathon Voter Season 6, Dedicated Voter Season 7, Marathon Voter Season 8, Star Voter Season 9 aka Clouds Without Water

I've only seen a couple of awful ones. Most are just kinda boring. Good enough for a homebrew, but nothing exciting about them.


What a nice idea for an item. But the language is horrible.

Marathon Voter Season 6

Sean K Reynolds wrote:
Just remember, there are HUNDREDS AND HUNDREDS of items submitted each year, and only THIRTY-TWO make it to the next round. That means hundreds are rejected for various reasons. And even if only 25% are rejected for being really bad, that's still HUNDREDS of items that are... destined for infamy, so to speak. :)

Well I feel a little better, having previously thought that my chances were one in thousands.

Marathon Voter Season 6

I had one where I seriously loved both items. I stared at them for a long while and shook my head as I clicked the one I liked just a bit more than the other.

Hope you get the votes you need, other person!

RPG Superstar 2012 Top 16 , Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Dedicated Voter Season 9 aka motteditor

For me, it's the lack of formatting on a lot of them (followed by ignoring advice that have been made very clear by the judges) I'm finding most startling. Trying now to make sure I choose one of them at least, though.

Sovereign Court RPG Superstar 2009 Top 32 , Star Voter Season 6 aka TerraNova

I can only re-iterate what previous posters have said - greatest respect for the judges. :) They did a hell of a job in previous years, and deserve a break. After all, they did have to pre-screen the items still.

RPG Superstar 2013 Top 16 , Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Dedicated Voter Season 9 aka Darkjoy

OK, noted. Will stop using the neither button. But I already hate voting for sub-optimal entries :<

Star Voter Season 6, Star Voter Season 9

Darkjoy wrote:
OK, noted. Will stop using the neither button. But I already hate voting for sub-optimal entries :<

for every bad item you vote for against a worse item, remember that a better item will soon be compared to both, and get both votes. That's the way the system works.

RPG Superstar 2011 Top 32 , Marathon Voter Season 6, Dedicated Voter Season 7, Dedicated Voter Season 8, Dedicated Voter Season 9 aka surfbored

This method of voting is starting to feel a little unfair, so I'm hoping it all comes out in the wash...

When I have two really terrible items showing, I must give a vote to one that doesn't really deserve a vote. Then, I'm served up two items that both deserve a vote, but only one gets a vote.

Now a bad item has one more vote than a far superior item.

Is there a way that this eventually balances out? (Please forgive my ignorance on the voting logic that is going on behind the scenes.)

RPG Superstar Season 9 Top 32 , Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Marathon Voter Season 9 aka theheadkase

If we press the "Report this item for breaking the contest rules" button do we get to submit a reason as well?

Assistant Software Developer , Star Voter Season 7

Jerry Keyes wrote:

This method of voting is starting to feel a little unfair, so I'm hoping it all comes out in the wash...

When I have two really terrible items showing, I must give a vote to one that doesn't really deserve a vote. Then, I'm served up two items that both deserve a vote, but only one gets a vote.

Now a bad item has one more vote than a far superior item.

Is there a way that this eventually balances out? (Please forgive my ignorance on the voting logic that is going on behind the scenes.)

When you vote for an item, you aren't saying "this item is good", you're saying "this item is better than that item". Beating a bad item doesn't mean as much as beating a good item.

In short, we're being smarter than just counting who got the most pluses.

Paizo Employee PostMonster General

You're not voting for or against an individual item. You're just telling us that "this item is better than that one". We have some complicated math that we're going to do that combines all those pairwise comparisons into one giant "this item is better than that item, is better than that item, is better than that item" chain. The judges will then use that chain, starting at the top-ranked item and working their way down, to select the final top 32.

If you click "report this item" you are not given a chance to provide a reason. If an item breaks the rules the judges can figure out what rule it's broken.

Dedicated Voter Season 6

This is kind of fun! But then, it has only been a couple of hours.

Major kudos for the folks who´ve taken it upon themselves over the past 5 years, definitely.

Marathon Voter Season 6

Pendin Fust wrote:
If we press the "Report this item for breaking the contest rules" button do we get to submit a reason as well?

No, and if you do that you can't vote for the other one.

If you see one really flagrant rule violator next to a really good item, what should you do?

Scarab Sages Marathon Voter Season 7

I notice that some of the items I voted "neither" for, do in fact show up again, allowing me to vote for a better item against it. So that sort of works out that way, but I duly note I should vote for the slightly less bad of two items if there is one.

And I have had one I reported because it clearly did not follow the template provided in any fashion.

Marathon Voter Season 6

There are a lot of items that don't have the price of the item as 2x the cost to craft it.

Is that a "breaking contest rules" issue? Or is only what was specified in the FAQ grounds for that vote?

RPG Superstar 2012 Top 16 , Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Dedicated Voter Season 9 aka motteditor

No, that's a design mistake, not a violation of the rules. Factor it in as you would missing coding or what have you.

Star Voter Season 6

There are some that are like that in the core rules, so don't make it too much against them. If it's off by a good margin, then yes, but if it's just like a hundred gold or so, it's not a big deal.

Marathon Voter Season 6

It's not even close, and some people actually reversed it, so it costs twice as much to craft an item than to buy one on the streets.

Assistant Software Developer , Star Voter Season 7

That's bad design, and a Pathfinder rules violation, but not a Contest rules violation. Judge it, but it's not a disqualification.

Paizo Employee Chief Technical Officer

inverseicarus wrote:
Pendin Fust wrote:
If we press the "Report this item for breaking the contest rules" button do we get to submit a reason as well?

No, and if you do that you can't vote for the other one.

If you see one really flagrant rule violator next to a really good item, what should you do?

Report it. It doesn't really help all that much if the awesome one beats one that gets dropped out of the competition.

RPG Superstar 2011 Top 32 , Marathon Voter Season 6, Dedicated Voter Season 7, Dedicated Voter Season 8, Dedicated Voter Season 9 aka surfbored

Now I'm embarrassed for "complaining" so quickly. If I had waited until after voting for several items, the pattern would've become clear to me, and now I get it.

After voting for roughly 4 hours last night, I really started to get the feel for how the contest is working and I have to say, it's genius!

Kudos to the Developers and another hearty pat on the back for the Judges. I can't believe what you must've endured for all the previous years.

...Back to voting!


Yeah, after reading about a dozen of these, I agree - Sean, Clark, and Neil should be fast-tracked for sainthood.

RPG Superstar 2009, Contributor

Wild Gazebo wrote:
My respect for the judges from the prior years has increased a thousand-fold.

/hat tip

Spoiler:

See? We told you guys it's hard to explain unless you've seen the other side of the curtain. Now you know what it's like when we find the really good submissions that get us excited. And now you know why we sometimes have to vent.

Scarab Sages Marathon Voter Season 7

inverseicarus wrote:
It's not even close, and some people actually reversed it, so it costs twice as much to craft an item than to buy one on the streets.

Reversing it is a pretty easy mistake to make - basically its putting the cost in the price and the price in the cost and then not realizing you got them backwards. Its one of those errors that when you do it, you don't notice it because the math is all right and you are fixated on other things.

Sczarni RPG Superstar 2012 Top 32 , Champion Voter Season 6, Champion Voter Season 7, Champion Voter Season 8, Champion Voter Season 9

Just hit 300 pairs. Many great names. Quite a few have caught my eye. The more items I look at, the more I wonder at the sanity of the Judges for volunteering to look through the items the previous years willingly.

Great job to all past, present, and future judges!

Owner - House of Books and Games LLC , Marathon Voter Season 6, Star Voter Season 7

Gary Teter wrote:
You're not voting for or against an individual item. You're just telling us that "this item is better than that one".

I think a lot of people don't quite get that; instead they think they're voting for one of the items to WIN.

Now, if the buttons said "Lazard's Ladle is better" and "Fu-Manchu's Mansion of Fu is better" there might be less confusion.

Edit: Again with the typing 'thing' instead of 'think!'

Marathon Voter Season 6

gbonehead wrote:
Gary Teter wrote:
You're not voting for or against an individual item. You're just telling us that "this item is better than that one".

I think a lot of people don't quite get that; instead they thing they're voting for one of the items to WIN.

Now, if the buttons said "Lazard's Ladle is better" and "Fu-Manchu's Mansion of Fu is better" there might be less confusion.

I was going to suggest the same thing. You vote for things you prefer to win. If the buttons are changed to the proposed, then it may be a bit clearer.

Paizo Employee Chief Technical Officer

1 person marked this as a favorite.
Thomas LeBlanc wrote:

Just hit 300 pairs. Many great names. Quite a few have caught my eye. The more items I look at, the more I wonder at the sanity of the Judges for volunteering to look through the items the previous years willingly.

Great job to all past, present, and future judges!

Perhaps now more folks will understand why the judges don't provide feedback on every item.

RPG Superstar 2012 Top 16 , Marathon Voter Season 6, Marathon Voter Season 7, Dedicated Voter Season 8

1 person marked this as a favorite.
Vic Wertz wrote:
Thomas LeBlanc wrote:

Just hit 300 pairs. Many great names. Quite a few have caught my eye. The more items I look at, the more I wonder at the sanity of the Judges for volunteering to look through the items the previous years willingly.

Great job to all past, present, and future judges!

Perhaps now more folks will understand why the judges don't provide feedback on every item.

I suspect more folks will understand a whole lot of things a lot better.

RPG Superstar 2009, Contributor

2 people marked this as a favorite.
Vic Wertz wrote:
Perhaps now more folks will understand why the judges don't provide feedback on every item.

Or how crazy they are when they try to... ;-)

Dark Archive

1 person marked this as a favorite.

Echoing the respect to the judges.

I've seen some neat evocative ideas, some terrible ideas, more spells-in-a-can than I care to remember (some decent items, actually, I think I'd rather read an interesting spell-in-a-can than a terrible 'original' idea!), an entry or two trying far too hard to do too many things, some slickly professional stuff and a few that cause the former editor in me to die a little more.

I think I've only 'neithered' one pairing. It was a real Sophie's choice.

Community / Forums / Archive / Paizo / RPG Superstar™ / General Discussion / A New-Found Respect For The Judges All Messageboards

Want to post a reply? Sign in.
Recent threads in General Discussion
DesignFinder