Appealing to the public vs. appealing to the judges


RPG Superstar™ General Discussion

Star Voter Season 6, Star Voter Season 7

One of the things I struggled with when designing my wondrous item this year was the dynamic between appealing to the public vs. appealing to the judges. I did a lot of research, and here's what I found (bear in mind this is my opinion).

Public voters seem to like:

- Gonzo more than non-gonzo (to a certain degree)

- Combat focused items more than roleplay focused items

- High power/lvl items more than low power/lvl items

- Unique/obscure item names vs more mundane names

Judges seem to like:

- Less gonzo

- Combat oriented items

- Low lvl items

- Mundane, practical names that tell you what the item does

Personally, as a voter, I lie somewhere in the middle. Items with names like Beardforge or Night Monarch Vardo really jumped out at me, and made me want to really take the time to find out what they were and what they did. I appreciated the risk-taking, slightly gonzo items, and tended to vote for them more often. However, I am a sucker for a good low lvl item, even if it's a bit mundane, and tended to upvote these more often when they were paired against a high-powered item. I also appreciate a good rp item, and favored those over combat-focused ones.

How do you guys tend to vote?

RPG Superstar Season 9 Top 32 , Star Voter Season 6, Marathon Voter Season 7, Dedicated Voter Season 8, Marathon Voter Season 9

Last year was my first year to vote and keep up with the contest. I was already a member, but my best friend entered so I took to voting.

When voting, I normally looked at the name last. I was really curious as to what the items did. I had no preference to combat and out of combat (though I did notice an overwhelming number of combat items). I will admit that my eyes would glaze over long stories and fluff. I like to know a small bit of what the item looks like, how it works, and what it does. As a GM, items with lots of background stories are a turn off at times because my players read way too much (and we all take turns GMing) and things get confusing when an item's backstory is changed from one everyone knows.

Other things I considered wen voting: pricing, availability by level vs powers gained at that level (if you get a great flight-based ability before level 5 from a cheap item, it is a no go for me), and was the item something new and different and of use. This last item was a biggie at times. I saw some items last year similar to items already available but switched to a different slot or offering only a small degree of change. I really enjoyed items that rather than give spell bonuses, instead allowed for players to do something different. I also liked the items that aided in areas where there is little other bonuses from items, class features, and spells.

RPG Superstar 2015 Top 8 , Star Voter Season 6, Star Voter Season 7, Star Voter Season 8 aka DeathQuaker

Things like combat or power level weren't really a consideration in and of themselves. More as to whether what the item did was appropriate to its balance, cost, etc.

I do tend to prefer practical items, although I don't like "cheat" items -- there were a lot of things like "this makes it easier for a wizard to prepare spells" or some such that didn't appeal to me, and I would say in general, I disliked items that were really only designed for the use of one class. Something that might read "in the hands of X, this item also does X" but I prefer items that are useful for most characters. Certainly there are going to be exceptions (the game itself has some good class-specific wondrous items) to that but generally I prefer the universal use stuff.

I know I got tired of things like "this looks like x, but really it also does x, y, and z." I disliked overcomplicated things. Often a potentially good item was ruined in my book by doing too much -- it might have had one or two core abilities that really worked, but then they added on more and more or made the mechanics so complicated it looked both annoying and confusing to use. I tended to prefer something that was simple yet elegant in terms of what it did... I recall for example a pipe when smoked granted certain skill bonuses. It was just flavorful enough and wasn't overcomplicated and would definitely be useful in game.

Mind, I fall into the "overcomplicated/does too much" trap myself in item design, but that's probably exactly why it bugs me. :)

I also dislike pointless flavortext filled with "look, I own a thesaurus!" jargon. Beyond a brief description to help the reader picture it, the focus of the text should be on, in my opinion, what the item does and how to activate it, etc.

I also just generally evaluated general writing style, cleanness of text, and reasonably grammatical correctness and such. If presented with two items that seemed generally equally innovative, but one entry was better and more clearly written, and obviously more carefully proofread, that is the one that is going to get the vote.

RPG Superstar 2010 Top 32, 2011 Top 4 , Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Marathon Voter Season 9 aka DankeSean

1 person marked this as a favorite.

On the 'gonzo vs. non-gonzo/public vs. judge' issue, I actually think it's the opposite of what you're suggesting. I think the public voting makes it much harder for an item that's a bit on the crazy side to slip through.

Speaking as a fan of the moderately gonzo item I feel like I saw a lot of this last year: clearly non-joke, but still somewhat crazy item pops up, tickled my fancy, made my list of favorites. Then after voting ended and discussion started, I'd mention one of those favorites, someone else would say they liked it, and then a handful of other people would say 'Item X? I couldn't vote for that! Why can't people take this contest seriously!'

I think what I'm saying is that, while designing gonzo will win you a certain amount of goodwill from a subset of the voters, there's just as large a subset of voters that won't appreciate your design and will downvote accordingly. Wheras, back in the days when all you had to do was appeal to three judges, it was easier to have an item's quirks draw just enough attention to it that it stood out, and then the actual design merits could be examined. The public voting though, a quirky item runs the risk of a larger public with differing voting criteria, a number of whom might have little interest in quirk.

Just my observation based on the way last years voting went; this isn't to say that the actual quality of the top 32 changed significantly, just that I don't think that, say, a migrus locker (to use the granddaddy of all gonzo items) would have made it through a public vote as easily as it did a closed judging.

Star Voter Season 7, Star Voter Season 8, Star Voter Season 9

1 person marked this as a favorite.

This is a game! That's why humor shouldn't be an auto-reject. The most memorable moments in my years of gaming are either funny or swashbuckling (which is often funny). But catchy titles, good writing and interesting effects are what I think I vote on, not in that order.

RPG Superstar 2009, Contributor

3 people marked this as a favorite.

This is an interesting topic. And it actually got discussed a bit last year, as well...particularly by Clark as he was very diligent in examining the impact the crowd-sourced voting would have on which items got elevated into the Keep pile for the judges to examine. He even looked well beyond the Top 89 (i.e., the number of items which made the Keep pile) to see what kinds of items fell on the other side of that threshold, just to make sure any "diamonds in the rough" weren't getting left behind and that the judges were doing their due diligence.

So, here's some relevant stuff Clark had to say about the differences between the two audiences evaluating the items:

Spoiler:
Clark Peterson wrote:

Let me help you a bit with the likely difference between what the public may be looking for (thus, how to get voted into the top group) and what the judges are looking for (how to make the top 32 from that group).

The two groups (voters vs. judges) are, in my view, looking for different things. That, to me, explains why some items got voted into the top 89 but are not items we as judges would put in the top 32.

Let me try to explain. I think it will help. Neil and I have been saying this over and over for a long time: the voters are looking purely at items, often with an "I want that item!" view of what is good or bad; the judges, on the other hand, are looking at the designer behind the item. Because in the end we are not really picking items, we are picking the best 32 contestants for RPG Superstar and the goal of RPG Superstar is not to find a great set of items, it is to find a great set of potential new designers.

This explains, I think, why voters tend to like and vote in walkie-talkie items or items that are really useful to a player while the judges never really seem to advance those kinds of items.

That was one of my concerns about a public vote to sort out the items. But having seen the results, my fears are allayed. I think the community of voters managed to find the best items, or at least the ones we likely would have marked as keepers.

The bottom line, though, is that if it is sufficiently awesome then both groups will like it.

And here's some data Clark shared from the differences in the outcome of the popular vote versus how the judges ultimately ranked and elevated items into the Top 32:

Spoiler:
Clark Peterson wrote:
Just for the record, remember how we used the top 89 items from the public vote from which to choose the top 32? Well, without naming names (and they don't necessarily relate in public ranking to judge ranking), the top 4 contestants submitted items 7, 11, 37 and 45 from the public vote. Just in case anyone was interested. Based on the judges rankings, though, of the top 32 they were items 2, 4, 22 and 28. So once again I think this shows the judge's input is valuable. Though it is clear the public does a good job of sorting the keep pile, it still really needs judicial ranking by experts of the top 32 from that pile. Lots to learn and think about. I love this contest!
Clark Peterson wrote:
The top vote getter IS in the top 32. We just didnt agree it was the best item. We thought it was middle of the pack of the top 32. In fact, the entire top 8 vote getters are all in the top 32. 9 of the top 10 vote getters are in the top 32.
Clark Peterson wrote:
...And I just found a real serious keeper at 68. I think all three of us agree on it, too...

In fact, you may just want to check out this entire discussion thread and this one too from Clark on how the items were evaluated. Clark gives a lot of insight which might just help everyone as they participate in the popular vote this year.

RPG Superstar 2013 Top 32, RPG Superstar 2012 Top 16 , Dedicated Voter Season 6, Star Voter Season 7, Star Voter Season 8, Star Voter Season 9 aka Hodge Podge

I talked a little about this last year, but my method's developed since then. It basically boils down to:

Core Idea/Mechanics > Writing/Name > Formatting

Idea/mechanics are more important than the writing/name, and writing/name is more important than formatting. However, I notice them in reverse order. Generally, I'll choose:

  • A) A well formatted, named, and written item with a pretty good idea
    -over-
  • B) A terribly written, named, and formatted item with an excellent idea.

    I even have a crazy rubric idea!:

    With the above in mind, you can give each area a weight:
    - Idea/Mechanics = 4
    - Writing/Name = 3
    - Formatting = 2

    Have a 3x3 grid, with the rows being formatting, writing/name, and idea/mechanics, in that order. The leftmost column is contestant 1, the rightmost is contestant 2, and the middle is for ties.

    Check the appropriate box for the winner in each area, and unless they tied in all three, there should be a clear pick. Unless you wanna golden ticket one of 'em. ;)

  • RPG Superstar 2013 Top 32, RPG Superstar 2012 Top 16 , Dedicated Voter Season 6, Star Voter Season 7, Star Voter Season 8, Star Voter Season 9 aka Hodge Podge

    Err, but yeah, seems I sort of went off-topic.

    I'm fine with both serious and Gonzo. What really matters to me is "will this item make my game fun?" Ones that make the game fun for EVERYONE get a serious plus in my book (like Beardforge ;p).

    Low-level items tend to fill gaps in the established game space, while higher level items tend to build on top of the rules. Either way, if the designer found an undiscovered niche, that shows me they have talent.

    For names, something that is both informative AND cool is the best for me. I try to do both.

    Definitely combat-focused is the way to go, but if you can word it in a certain way that allows extrapolation into other areas of play, I think that's huge!

    Star Voter Season 6, Dedicated Voter Season 7, Star Voter Season 8

    Bryan Bloomer wrote:
    Items with names like Beardforge or Night Monarch Vardo really jumped out at me, and made me want to really take the time to find out what they were and what they did. I appreciated the risk-taking, slightly gonzo items, and tended to vote for them more often.

    Unfortunately, I think the beardforge turned out to be a dud with voters.

    But yeah, I'm (obviously) pro-gonzo all the way. Anything that makes me think the designer is going to wow me in the monster, encounter, or module rounds is going to get my vote.

    Star Voter Season 6, Star Voter Season 7, Star Voter Season 8

    Tom Forkbeard wrote:
    This is a game! That's why humor shouldn't be an auto-reject. The most memorable moments in my years of gaming are either funny or swashbuckling (which is often funny). But catchy titles, good writing and interesting effects are what I think I vote on, not in that order.

    That said, I think deliberately aiming for "the humorous item" is a slippery slope. What's funny and what's not is a subjective, to be compared to the objectivity of game balance and formatting. Gauging one way or the other on humor just distracts from whether it's actually a good item.

    RPG Superstar Season 9 Top 32 , Marathon Voter Season 6, Marathon Voter Season 7, Champion Voter Season 8, Marathon Voter Season 9 aka GM_Solspiral

    I call that line the yarn ball line...

    Grand Lodge

    1 person marked this as a favorite.
    Pathfinder PF Special Edition, Starfinder Roleplaying Game Subscriber
    Tom Forkbeard wrote:
    This is a game! That's why humor shouldn't be an auto-reject. The most memorable moments in my years of gaming are either funny or swashbuckling (which is often funny). But catchy titles, good writing and interesting effects are what I think I vote on, not in that order.

    Again, this isn't a contest of acception or rejection. Your humor item is simply going to have to be that MUCH BETTER than the non-humor item it's been paired up against.

    If I want jokes, I'll wait for the next April Fool's issue of Dragon or Wayfinder. If Paizo put out a book of Joke Items, I'd suspend the relevant subscription until it passed.

    Shadow Lodge RPG Superstar 2015 , Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Star Voter Season 9 aka mamaursula

    Last year I earned "Marathon Voter" because I lost my mind, I haven't even read all of the source books. I probably read more "Pathfinder" material voting than all my other reading combined.

    That all said, I am not fond of "gonzo", I really like practical things. I voted for a lot of things that didn't make it in the Top 32 and didn't like much of what made it in (no offense to all the great people and their hard work last year). I read the forums and followed as time allowed deep into the contest. I will endeavor to be more critical this year in parts of the items that I didn't really pay attention to.

    I think "appealing" to the judges vs the popular vote isn't really where you should focus, you should focus on making something you can visualize working in a game, maybe your game, and write it with the best balance you can to the best of your abilities. You should ask others what they think of it and consider all criticism, but in the end follow your gut.

    Regardless of how things turn out, you will learn something and you get to keep what you learn and apply it to the next thing you try to design.

    RPG Superstar Season 9 Top 32 , Marathon Voter Season 6, Marathon Voter Season 7, Champion Voter Season 8, Marathon Voter Season 9 aka GM_Solspiral

    On main post topic: public appeal contrasting with judge appeal, I'd lean toward public appeal.

    Make something for what you think would make the judges happy and the public doesn't like it, well then it gets sorted towards the bottom and never seen. There's also the matter of these are judges that to my knowledge have never judged Superstar before so you do not know what appeals to them per se, you do know they are part of the public.

    At the end of the day good design is good design, but good design isn't enough. If they wanted good designers there would be no need for the contest, they want undiscovered rockstars. You can work your way up but you either rock or you don't plain and simple.


    Tom Forkbeard wrote:
    This is a game! That's why humor shouldn't be an auto-reject. The most memorable moments in my years of gaming are either funny or swashbuckling (which is often funny). But catchy titles, good writing and interesting effects are what I think I vote on, not in that order.

    There's a fine line where humor is concerned, and most people honestly don't know when they've crossed it. Last year's beardforge was funny and it was kind of cool. I voted it up more often then not. There were other items out there that weren't as cool and the attempts to be funny fell flat, sometimes because they relied on stereotypes that could be offensive (I'll refrain from mentioning specific items as I don't want to start up bad blood before the contest starts). Most of them looked like they could have been found in the slush pile for Dragon magazine's annual April Fool's issue--in other words, cheap yuks and not much substance.

    It's like when people don't know the difference between an homage and a rip-off. "Bigger on the inside" items could be considered an homage. A blue box that's bigger on the inside and can transport you across space and time, however...

    Marathon Voter Season 6, Marathon Voter Season 7, Star Voter Season 8

    Some humor can be fine... but if your item makes use of a three stooges or Warner Brothers Cartoon reference (or even worse requires the player to use one to activate its comedic power) it is not going to make to the top 32, bottom line.

    Silver Crusade Star Voter Season 7

    Reminded of the wildly split opinions on Mike Welham's Monster Reformation Alliance. That may have been the most polarized the judges and public voters ever got, eh?

    And he got the gold medal. :)

    Grand Lodge Marathon Voter Season 7

    1 person marked this as a favorite.
    Mikaze wrote:

    Reminded of the wildly split opinions on Mike Welham's Monster Reformation Alliance. That may have been the most polarized the judges and public voters ever got, eh?

    And he got the gold medal. :)

    That one was an interesting one. His item round was also one of his weaker rounds, oddly enough.

    I'm not sure if there's a great takeaway lesson there, other than that you should enter even if you think that the item round will be your weakest, and that the judges' opinions may not always reflect the public's.

    Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Dedicated Voter Season 9

    3 people marked this as a favorite.
    Adventure Path Charter Subscriber; Pathfinder Rulebook, Starfinder Adventure Path, Starfinder Maps, Starfinder Roleplaying Game Subscriber

    I am taking a bit of a devils advocate stance for you all to consider. You should remove versus. The best items appeal to the public AND the judges.

    Think of it this way, the judges are seeking designers who design things the public demand and who make fun an inherent part of the game.

    Happy new year all and good luck to everyone in the voting.

    Liberty's Edge RPG Superstar 2012 Top 16 , Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Star Voter Season 9 aka Locke1520

    Anthony Adam wrote:

    I am taking a bit of a devils advocate stance for you all to consider. You should remove versus. The best items appeal to the public AND the judges.

    Think of it this way, the judges are seeking designers who design things the public demand and who make fun an inherent part of the game.

    Happy new year all and good luck to everyone in the voting.

    I don't see this as a devil's advocate stance...It's a wise stance.

    RPG Superstar 2013 Top 32 , Dedicated Voter Season 6, Star Voter Season 7 aka Standback

    Anthony Adam wrote:

    I am taking a bit of a devils advocate stance for you all to consider. You should remove versus. The best items appeal to the public AND the judges.

    Think of it this way, the judges are seeking designers who design things the public demand and who make fun an inherent part of the game.

    Precisely this. The best items will appeal to both; otherwise, there's something in it that's turning off one group or the other.

    Don't make any mistakes. Know why your item is cool. That should get you by both of 'em.

    Silver Crusade Star Voter Season 7

    Samuel Stone wrote:
    I'm not sure if there's a great takeaway lesson there, other than that you should enter even if you think that the item round will be your weakest, and that the judges' opinions may not always reflect the public's.

    I suspect this may change quite a bit with the larger pool of judges this year. Gonna be interesting to see how it differs from previous years!

    Community / Forums / Archive / Paizo / RPG Superstar™ / General Discussion / Appealing to the public vs. appealing to the judges All Messageboards

    Want to post a reply? Sign in.
    Recent threads in General Discussion