| 1 person marked this as a favorite. |
Hey RPGSS runners,
Are there any plans on releasing the top 100 items this year? I know last year finding out what kind of items made the top and comparing myself against them really helped me figure out what to do differently this year (and hopefully perform better). I would like the opportunity to do that again if possible.
That would be really nice...
| 2 people marked this as a favorite. |
We looked at them starting from the top of the voters' votes, plus a small number of additional items that I thought were worthy contenders (since I had looked at all of the items). Rest assured, by far the best chance to make the Top 32 was by winning as many votes as possible (all 4 of the Top 4 items by votes made Top 32, for example, and 7 more of the Top 20 by votes are either Top 32 or alternates). In fact, this year had the weakest amount of judge golden ticketing of any year. Other than submitting my few items for the other two judges to consider, I refused to give them any more advantage; if Owen and Liz didn't like them, they didn't advance. We required a strict majority for Top 32, with no golden tickets.
| 1 person marked this as a favorite. |
Top 100, top 110, whichever, doesn't matter. I'd just like to see the list so I could learn from them.
| 1 person marked this as a favorite. |
We looked at them starting from the top of the voters' votes, plus a small number of additional items that I thought were worthy contenders (since I had looked at all of the items). Rest assured, by far the best chance to make the Top 32 was by winning as many votes as possible (all 4 of the Top 4 items by votes made Top 32, for example, and 7 more of the Top 20 by votes are either Top 32 or alternates). In fact, this year had the weakest amount of judge golden ticketing of any year. Other than submitting my few items for the other two judges to consider, I refused to give them any more advantage; if Owen and Liz didn't like them, they didn't advance. We required a strict majority for Top 32, with no golden tickets.
4+7 of the Top 20 are Top 32 or alternates, which, put another way, means that 45% of the Top 20 by vote were still shot down by the judges. With this year having less gold-ticketing than in years past, is it then fair to say that even if your item was voted into the Top 20 in years past, you still only had a 50/50 (or less!) shot of making it to the Top 32?
Modules Overlord
|
Everything in the top 100 was looked at by at least two of the three judges, and everything in the top 40 was looked at by every judge. Everything that made it into the top 32 had 2 of 3 judges voting for it, and only one other item (which was an alternate) got 2 of 3 judge votes. I'm quite comfortable with those results.
Since we changed first round voting, it's the case that the judges' opinions are what determine the top 32 and alternates with the fan vote serving an advisory capacity, unlike every other round when the judges serve in an advisory capacity, and fans make the final determination.
I'm not 100% satisfied this is the best possibly way of handling this, but I wanted to get one contest under my belt before I tackled changes to voting. There are very real problems with any system that sorts 900+ items without asking a large group to through them all and rate them from 900 to 1. The current system is imperfect, but I'm happy with its end results. Obviously other judges would have likely picked slightly different set of items. That's going to be true of any subjective talent-based competition.
We really appreciate fans voting, so much so that we give acknowledgments to the ones who vote a lot. It's a huge help, and a crucial form of input. But if anyone would rather not take the time because the judges have the final say, I certainly understand. That's behind us this year, and we may make a change for next year.
If anyone has input they'd like us to consider on different voting options we could use next year I'm open to ideas, though I've already discovered there are things we don't have the time or resources to implement, and other ideas have complications that aren't obvious.
|
Modules Overlord
|
| 6 people marked this as a favorite. |
I'm not currently planning to release the list of top 100 items or make them open to public viewing. I very much want to keep the focus of the contest on the people who have all worked hard on two rounds, since everyone who passed round one has already turned in a map at this point.
| 3 people marked this as a favorite. |
Owen,
This is not a fan friendly move.
It's not about distracting from the top 32 it's about investing in the future of the contest by engaging the folk that were near misses. Finding out you're top 100 is validation. Finding out you missed the mark with the public softens the blow as you can say... maybe the judges didn't see my item. Makes you think about making something with more mass appeal the next year and maybe looking at those items and trying to emulate what they did right.
This bit with Mark sorting out favorites in advance of the vote also sounds bad. Basically it sounds like 1 judge read ahead of everyone, formed opinions before the public finished sorting and those opinions set the tone of how items were viewed.
What the heck is the point of asking your fans that buy your products vote and sort the entries if one person's opinion is going to set the tone of how items are viewed. Honestly the judges should be blind to the items until the sort is done. Or better yet, release the top 100 and let the public vote in the top 32 and alternates. That way personal tastes and biases are removed.
There's nothing wrong with change and some of the changes have been awesome (changing up R1, the Map round in general, returning to judge feedback in the CMI thread.) With some things you are throwing the baby out with the bathwater.
~End rant
***edited to heck
| 3 people marked this as a favorite. |
The top 100 list helped me last year.
In the three years of player voting, my progression has gone like this -
2013: Didn't get culled.
2014: Made top 100
2015: Alternate
That top 100 was important to me, it let me know I'd entered something the voters liked and that I should keep trying. I'd enter anyway, but I was more motivated to work at it this year by knowing I'd come close last year.
|
Modules Overlord
|
| 2 people marked this as a favorite. |
I certainly did not intent to skip a blog post thanking voters for their efforts. I simply didn't see that was something we've done before. The voters efforts are crucial, and this is absolutely a failing on my part.
My sincere apologies to everyone who voted. It was not my intent to fail to give you the acknowledgement you absolutely deserve. I'll look to improve on this issue.
|
Modules Overlord
|
What the f$%k is the point of asking your fans that buy your products vote and sort the entries if one person's opinion is going to set the tone of how items are viewed. Honestly the judges should be blind to the items until the sort is done. Or better yet, release the top 100 and let the public vote in the top 32 and alternates. That way personal tastes and biases are removed..
I'm going to start by asking you to keep an eye on your tone. Replacing letters in a word with symbols does not change what you are saying, and that's inappropriate language for the Paizo boards.
As it happens, everything Mark thought about the items he looked at was kept to himself until we looked at the list of top items. He careful put "**Nurtal place holder opinion" in every entry, rather than color our initial thoughts. That said, the judges have always had access to all the items as soon as they are turned in. They have never been limited to just a subset, and with golden tickets could always have pulled anyone out of any rank. We pretty well all agreed (the 33 people who got 2/3 votes made our top 32 and the first alternate), so golden tickets weren't needed.
I'm not 1005 satisfied with this process myself. I think your idea is a fine and reasonable suggestion for next year, and I'm going to add it to the list of things to consider.
As for the judging process - no one failed to be considered, and the judges picked a top 32 + 4 I'm quite happy with. It sounds like opening up the transparency more may lead to the judges who acted under my instructing being criticized, and I am unwilling to expose them to that.
It was never my intent to release a top 100 list this year. I honestly think the entire round 1 voting process needs greater consideration and expiation up front (even the judges weren't 100% sure how we were handling the internal issues when we started), but at this point those are questions for next year.
Replacing letters in a word with symbols does not change what you are saying, and that's inappropriate language for the Paizo boards…....
….I'm not 1005 satisfied with this process myself.
Sorry. Couldn't help myself. You still can edit… :)
I honestly think the entire round 1 voting process needs greater consideration and expiation up front (even the judges weren't 100% sure how we were handling the internal issues when we started), but at this point those are questions for next year.
I think Round 1 definitely needs to change in how the Top 32 are arrived at.
| 5 people marked this as a favorite. |
Owen,...This is not a fan friendly move.
Frank,
This isn't exactly a fan friendly move either. Publicly criticizing the decision (as opposed to sending a private message or email) isn't the best way to go about trying to improve the contest as you're seemingly attempting to do.
Also, I'd quibble with more than a handful of the assertions you've made.
It's like skipping the blog post thanking the voters for their time and energy and acknowledging the Champion voters.
This may be a valid point. Taking the time out to thank everyone is just the gracious thing to do (and for many of the reasons you've cited). They've certainly been thanked quite a lot in prior years. Those who gain the Champion voter tag are almost always repeat voters. So, I'd assume they don't need qualifying year after year. But, as you say, it's a simple thing. I seem to recall at least somewhere among the various RPGSS threads that at least some of the judges thanked the voters. That said, in the grand scheme of things, this is kind of small potatoes in terms of being rant-worthy (but that's just my opinion).
It's like not releasing the rules a little early.
I'd disagree here. I've always thought that releasing the rules early went too easy on the competitors. Granted, it can gin up further excitement from everyone else eager to try their hand at a design challenge. And, as you said, lots of people start throwing up the "What I Would Have Submitted" threads. But, honestly, I've always felt those threads distracted from the actual contestants. At best, I get what you're saying...i.e., that they can foster much greater interest in those who missed the cut, encourage more folks outside the contest to hone their skills, and even keep more voters around to support the competition. However, at worst, I think it serves as a distraction. In many ways, it smacks of a desire to take the attention off the competitors in order to look in on what others have produced. I think it's great if people want to try their hand at these design challenges. But there are many more outlets for doing that than lining up beside the active competitors and drawing attention to yourself (presumably in an effort to compare your skills to those who won their way past you to take their place in the competition). And, ultimately, to bring this full circle, I think giving the rules too early runs the risk of doing the actual competitors a disservice...i.e., it fails to put them into an adequate crucible to better identify those with true Superstar talent.
Releasing that list also costs nothing and encourages repeat entries....It also gives everyone data on the difference between marketable items and items that pass professional muster.
Personally, I think there's no shortage of repeat entries for the contest. The number of returning veterans year after year is indication of that. Additionally, posting the Top 100 isn't a necessary component of giving everyone data on the difference between marketable items and items that pass professional muster. That much should be clearly evident to those who do their homework. And, relying on a Top 100 post actually undermines getting everyone to get out there and do that homework. You actually learn better in the doing of that research for yourself than the telling of it by someone putting together such a list...and that's primarily because the list itself is completely void of any context. Thus, it's far better to refer to the "Critique My Item" thread for that kind of insight. Or, the commentary on the actual competitors' Top 32 submissions. Or, even the process of looking back over actual published material and identifying which items among them are more representative of Superstar-caliber designs as opposed to "good enough for a book of..." whatever.
It's not about distracting from the Top 32....it's about investing in the future of the contest by engaging the folk that were near misses. Finding out you're Top 100 is validation.
And yet, it actually is a distraction from the Top 32. There are also plenty of other ways to invest in the future of the contest by engaging others who missed the cut, evidenced by all those other opportunities mentioned above. Finding out you're in the Top 100...or that you made the cull...or, in the past, that you made the Keep folder may certainly encourage folks to keep trying. But the experiences of perennial competitors like Mike Welham and Steve Helt, both of whom labored for years and kept missing the cut, but eventually went on to win the whole thing, should already offer object lessons about the importance of sticking with it. No one should need a Top 100 validation to encourage them to do that. And, if you do require that level of validation to maintain your inner desire to keep going, I'd question your staying power when the harsh critiques ultimately come down on you even if you did make it into the competition.
Instead, here's what I believe drives the need for validation of making the Top 100. It serves two purposes...neither of which I believe are in the best interests of the contest or those seeking such validation. The first is that it's a natural part of the cycle of depression that people go through when they try out for something and then find out they don't make it. It takes a series of stages including: Denial, Anger, Bargaining, Depression, and eventually Acceptance. For someone to pursue validation so strongly via the posting of the Top 100, they're still denying that they weren't good enough to qualify for the Top 32. Or, they're angry and they want to see that they at least made the Top 100. Or, they're still trying to bargain their way through by convincing themselves that at least if they made the Top 100, they were really Top 32-worthy and it was only because someone else missed their awesomeness that they weren't upvoted enough or chosen by the judges. You've got to get past those three stages to make your way through the depressing reality that you're not in that Top 32. And, then, you reach the fifth stage of accepting that and beginning the up-cycle of recovery so you can regain confidence and try your hand again.
I believe the second purpose in wanting to see the Top 100 as a need for validation is that those who ask so diligently for it simply want more insight into "what the judges are looking for" and "what the voters are looking for" because they want an added competitive edge for the next time around. It's not simply the encouragement and validation they want. It's insight...an ability to go back and examine those Top 100 item designs to mine them for indicators of what will capture the voting public's interest...and to compare them to the Top 32 to determine what factors in those items allowed them to be pulled up from the Top 100 for inclusion in the contest...and all in the hopes of bettering your own design for next time. But is that really necessary? It's pretty clear that you can get a sense of what it takes to make the contest by simply reviewing the Top 32. And there are plenty of other ways to get design insights from the "Critique My Item" thread, and so on. It's not necessary to post a Top 100...or announce all the items in the Keep folder...to give people that. And, in fact, the celebratory hoopla which would surround such an announcement would most certainly distract (and, in some cases, even detract) from the Top 32. So, in the interests of supporting and encouraging the actual competitors in the contest, it's best to keep the focus on them rather than the Top 100.
Not releasing the data looks bad.
I actually think it looks fine. And I can say that both as a former-judge and a former-competitor. So, I know both sides of this thing. Why would anyone even think it looks bad anyway? Are accusations that Paizo is "hiding something" going to follow simply because they don't show you every item that missed cut...or by how much they missed it? That's patently ridiculous. There are vast numbers of contests which don't give you any insight into any other submissions except those selected to compete. And yet, Paizo gives you the opportunity to see so many items during the public vote now that the transparency is huge by comparison. Yet, that's still not good enough? They've got to showcase the Top 100 in the midst of the Top 32 trying to compete in the actual competition or they'll look "bad?" That just doesn't ring true to me at all. And if anyone feels that way about it, I think it's a poorer reflection on them than Paizo and the Superstar judges.
Every time this contest has moved toward transparency it has been made the contest better. The excuse that it's all about round 2 is flimsy. You could have released it a week ago... No distraction no harm.
This is laughable to me. And here's why. I've been involved with the contest enough times to see every move towards "transparency" which you (and others) value. And yet, every single time, it's not good enough. People still want more transparency. Show us the raw comments from the judges. Show us the Keep folder. Give us in-depth critiques on every submission. None of it is ever good enough. And the contest already has more transparency than any other I've ever seen or been involved with.
The excuse that it's "all about Round 2" isn't flimsy at all. Paizo, the judges, the actual competitors, and I believe the vast majority of voters actually want the competition to be focused on the Round 2 designs and not the Top 100. Could such a list have been released a week ago? Possibly...assuming the Superstar judges or technical team set aside all their other work and made that a priority when it doesn't need to be. The purposes of the contest (i.e., the selection of 32 competitors to run the gauntlet) has already been served. There's literally thousands upon thousands of messageboard posts dating back over 6 or 7 years now which are already devoted to fostering continued participation in the contest by those who missed the cut. Suddenly posting up a Top 100 isn't going to somehow raise that to a much-needed higher level somehow.
I'm of the belief that the need for validation (whether as part of the Top 100, the Keep folder, or whatever) is a self-serving goal for those who either want to see themselves within it (because they need it for motivation) or, because they want a greater insight into how to improve themselves and their chances to crack the Top 32. Firstly, if you're a Superstar-caliber competitor you should never need that kind of validation to keep going. And, secondly, you've already got a plethora of tools available to you to improve yourself before your next go.
This bit with Mark sorting out favorites in advance of the vote also sounds bad. Basically it sounds like 1 judge read ahead of everyone, formed opinions before the public finished sorting and those opinions set the tone of how items were viewed.
Now this is sounding like sour grapes...or some kind of attempt to look for ways to invalidate the choices the judges made in excluding you by quibbling over how they went about their business. Like Mark, I too read ahead when I was a Superstar judge...not because we had public voting at that time, but because I'm a completionist and wanted to assess everything. But, even Clark Peterson did read beyond just the items that the public up-voted into the Top 100 (or 189?) the first year the judges let the public sort the items for them. He didn't do that to form opinions ahead of their voting so he could set the tone of how items were viewed. He did that to quality-assure the process...to ensure there wasn't an item beyond the Top 32...or the Top 100...or the Top 189...which got excluded from the public vote, but which the judges might still have considered. And do you know why the judges do that? It's because the public vote can often differ quite substantially from how the judges themselves would have voted if they were doing the sorting on their own.
It's pretty common knowledge that not everyone playing along at home and participating in the public vote are doing so with an eye towards evaluating the submissions the same way the experienced, veteran judges would. Many vote for the item rather than the designer, valuing it for their home game or their PC, and caring not at all for the design choices and nuances the actual competitor wove into it. Many also vote for items which have some pretty serious, game-breaking flaws to them, because they're not as trained up on the rules as the judges. They just vote for what seems cool rather than what's done well. Or, they miss the promise of what a particular design idea or niche tells the judges about a designer as opposed to the item itself.
In the end, there are enough differences between how the judges look at things and how the public voters look at things, that the contest deserves a thorough sanity check. Clark provided that when he looked beyond just what the public sorting had brought to the judges queue for their consideration. I believe Mark's efforts were very much in the same vein. He may have looked beyond the Top 100 and found an item or two still worthy of consideration. Whether or not those were similarly championed by his co-judges is unknown to us unless they choose to share it. But, I for one value the fact that Mark would take on that task of "reading ahead" and quality-assuring the process. If he found an extra few "diamonds in the rough," more power to him. I'm sure the other judges evaluated them alongside him with the same care and consideration they gave all the rest. And that's all you really need to know about that.
What the f$%k is the point of asking your fans that buy your products vote and sort the entries if one person's opinion is going to set the tone of how items are viewed. Honestly the judges should be blind to the items until the sort is done. Or better yet, release the top 100 and let the public vote in the top 32 and alternates. That way personal tastes and biases are removed.
This is a colossally bad idea. The fans (collectively) are not the best qualified to determine the Top 32. I think that alone is evident from the fact that there's a 45% hit rate on what the public up-voted into their Top 32 and what the judges ultimately settled on. By asserting what you've suggested here (or hinted at), you're implying that you know better than the judges as to what deserves to be in the Top 32. Or, at the very least, you're suggesting that you trust the public voters to collectively establish that better than the judges. I'm sure that would work out better for someone who wants only one audience to win over with their submission, but that's absolutely not what's best for determining who to select for the Top 32. You know how I know? Because the contest is working just fine at identifying very valuable freelancer talent for Paizo from the Top 8 and up (and sometimes even from the Top 8 thru the Top 32). And those individuals, for 6-7 years running now, have always been selected by the judges rather than the voters. The voters certainly have their role to play in determining the actual Superstar by having them determine who advances. But inclusion in that Top 32 is something you have to convince an experienced judge to provide you. Not the masses. And that's how it ought to be.
~End rant
~End counter-rant
--Neil
I'm going to start by asking you to keep an eye on your tone. Replacing letters in a word with symbols does not change what you are saying, and that's inappropriate language for the Paizo boards.
Acknowledged and edited. I'm a passionate person and this contest means a great deal to me. An offhanded comment made by Clark Peterson was enough to encourage me to try my hand at freelancing. Wolfgang Baur's Kobold Press blog was the first place I cut my teeth as a freelancer.
These judges inspired me to take bold steps toward making my dreams a reality. Writing helped me get past depression, helped me quit smoking, and helped me find the motivation to start running.
A little bit of encouragement was all I needed to put myself on a better path. I try to pay that forward in this competition.
I'm not 1005 satisfied with this process myself. I think your idea is a fine and reasonable suggestion for next year, and I'm going to add it to the list of things to consider.
Trust the fans to not let you down.
As for the judging process - no one failed to be considered, and the judges picked a top 32 + 4 I'm quite happy with. It sounds like opening up the transparency more may lead to the judges who acted under my instructing being criticized, and I am unwilling to expose them to that.
How does releasing the list of item names without ranking them expose your judges to more criticism then they already get? The top 32 is a subjective selection process that is always going to draw criticism. Criticism is not a bad thing unless it falls into abuse, I'd like to think this community has shown itself to be better than that.
Some of us are very outspoken people and some of us are occasionally spirited with our opinions. That's a good thing as that is an engaged community that cares. Ambivalence is the true threat, the worst thing that can happen to this contest is people not speaking up and silently voting with their feet.
I'm not going to push further on the top 100, I'll let my comments on that stand. I have items and maps to critique...
| 2 people marked this as a favorite. |
I'd like to add a small comment -- while I agree the focus does need to be and should be on the Top 32 contestants (and I feel it has been in past years and I'm guessing it will be this year);
there is only so much that can be done here on the forums related to those few people.
In trying to critique items, I have put off even bothering to comment on the Top 32 + alternatives because when I read through the comments of the other 20+ people that have commented on each, + the judges comments, I basically have nothing left to add. And after reading the first dozen, I don't need to see the rest really.
Comparatively, those voters you want and need back for the later rounds can be drawn back by these other things going on. While I don't care if the Top 100 is released (Neil, you have an awesome discussion of why it shouldn't --- kudos to you), I always loved coming back over the days, weeks, and months of the competition to see what people had to say about my item in the Official Critique Thread (made so much easier by the efforts of people like Garrett --- thanks man).
Even commenting tonight on maps, I realized I had almost nothing to add that the judges hadn't already brought up, because... they're maps. Trying to remove critique on their artistic merit leaves me just with a million questions for the designer... which they're not allowed to answer till after voting is done so... back to critiquing all the items of everyone in the Official Thread again I suppose!
Owen, regardless of everything else, I appreciate how you've at least responded with a long and thoughtful series of responses to someone raising the issue and you sound earnest in your interest to make changes to the future of RPG Superstar --- which is nice to hear, even if you and whomever else is in charge decide not to change a thing.
| 2 people marked this as a favorite. |
I likewise was looking forward to seeing the Top 100, and I don't think it necessarily detracts from the advancing contenders.
I'm fine with not seeing it this year, and we can collect data to see if it did increase engagement on the other rounds of the contest (beyond average growth in engagement) or if there's enough activity to go around for both.
| 2 people marked this as a favorite. |
Honestly, I think Mark Seifter's diligent feedback provided a very awesome alternative to the top 100. His and voter feedback strike me as a better measurement of how well your item faired and how you can improve than some arbitrary top X items list.
I do feel that the strongest argument for showing the list lies with less golden ticketing since getting those few more votes than the other item in the top 50 is more important than simply making an item great enough to catch the eye of the judges. However, I do agree the arguments against showing the list outweigh this.
| 1 person marked this as a favorite. |
I disagree rather strongly about whether someone "should" or "should not" need validation to keep at it. People need what they need, and it's different for everyone.
I have no doubt whatsoever that in some cases it can make the difference between someone continuing in a creative endeavor and giving up. I've seen it happen any number of times, with people that went on to become professionals in whatever particular field.
I've been around various creative worlds for most of my life, to a greater or lesser degree. Music, film, writing. I've encountered the idea that 'if you need validation, you aren't going to make it anyway' in each of them, but in my observation I've seen it proved wrong over and over again.
There are some people who are going to make it no matter what. Most people who are going to hit it really big are apparent early on. They have an intense internal drive that's integral to their personality. Those people may not "need" validation (though they quite often crave it intensely).
But the business isn't just superstars, there's a lot of rank and file pros that make the industry work. And some of those people actually do benefit from validation. It really can be the difference in making it or giving up. I've seen it over and over.
| 3 people marked this as a favorite. |
A longer explanation of what I mean, with special thanks to Owen, Neil, and Garrett...
Hey, Lucus...great points. Here's a short response to some things you brought up, which I think you should consider:
In trying to critique items, I have put off even bothering to comment on the Top 32 + alternatives because when I read through the comments of the other 20+ people that have commented on each, + the judges comments, I basically have nothing left to add. And after reading the first dozen, I don't need to see the rest really....I always loved coming back over the days, weeks, and months of the competition to see what people had to say about my item in the Official Critique Thread....
Consider those two statements you've made there. You always loved coming back over the days, weeks, and months of the competition to see what people had to say about your item in the official "Critique My Item" thread...and yet, you also don't think you have anything left to add to the discussion of the Top 32's submissions? Much like you, they too love coming back to see what else is said about their work...in every round. So, even if all you have to offer is a "Me, too!" in agreeing with the judges or some other commenter, it helps them validate the feedback they received...i.e., they get a sense of 1) how many people are taking the time to respond to their work, and 2) how well that work was received. So, if you enjoy all that commentary you get from people in the "Critique My Item" thread, consider passing on that same experience to the Top 32 by giving that to them, as well. Every bit of feedback is absorbed by these designers. And, quite honestly, they need it to grow over the course of the competition. So, don't discount what your voice can add to the comments on their submission threads. I assure you, as a former-competitor, we tremendously value it.
Even commenting tonight on maps, I realized I had almost nothing to add that the judges hadn't already brought up, because... they're maps. Trying to remove critique on their artistic merit leaves me just with a million questions for the designer... which they're not allowed to answer till after voting is done so... back to critiquing all the items of everyone in the Official Thread again I suppose!
Ask those questions anyway, because even if they can't answer (right away), it gives them insight into the kinds of questions they left unanswered with their design. And, ultimately, they'll grow in knowledge and experience for how to tailor their designs based on the questions they see people asking about their work. Sometimes, even if you're just a fly on the wall observing a discussion like that, you can learn so much from it. So, the gag order on the contestants isn't a reason to withhold your commentary or questions about their design. It's still valuable feedback.
My two cents,
--Neil
On the topic of voting and top 32 selection by the judges-
As long as the contest uses both voting and relatively free-ranging judges to select the top 32, there's always going to be a little resentment from some of the voting population. People are having their preferences overruled, and in general people don't like that, it's just how people are.
I'm not sure it's a world-ending issue, exactly, but neither do I see it going away.
I love voting, as is obvious by my voter tags. I love seeing nearly all the submissions and evaluating them for myself. But I'm not sure it's serving the purpose it was originally brought in to serve.
Originally the idea was that judges weren't going to have enough time to read all the items. But if there are judges reading all the items with an eye toward rescuing contestants the voters wrongly rated, then are the voters really necessary?
If it were my contest, I'd go one of two ways with it. I'd either make the top 32 voted items in by default and tell the judges they'd have to make very strong cases why any item in that set should be overruled. In other words, I'd make the voter preferences carry much more weight.
Or... I'd eliminate the voting on items and leave it entirely up to the judges. I'd require that at least one or more of them read every item and sort them into a keep folder for all judges to review and rate. In other words, pretty much the old system. That system worked, and it could work again.
It's the hybrid with loosely defined boundaries that causes whatever level of resentment exists.
As I said, I'm not sure there's a deeply serious problem, but it is something that keeps coming up, so maybe it should be looked at more closely.
RPG Superstar 2014 Top 16
,
Marathon Voter Season 7, Marathon Voter Season 8
aka Jrcmarine
|
| 5 people marked this as a favorite. |
I agree with Neil regarding comment. Last year you could tell who was doing well in each round by the number of posts to their thread. The less posts, the less likely they were to advance. I received mixed reviews on my item and was middle of the pack with hits. My monster was pretty popular, despite his flaws, and my thread saw lots of hits. My location wasn't as popular and you could see that in the number of hits. Those that did comment were great because it provided me with insight, even if the poster was merely validating what the judges said. It still was nice to see people engaged in the contest.
There was a lot of voter fatigue last year and I am closely watching this year to see if voter fatigue sets in earlier and harder than last year. Even though I haven't advanced this year, I am going to keep an eye on the contest and try to offer comments throughout. I know I appreciated ALL feedback that I received throughout. And I know it was a little disheartening the deeper I got into the competition to see less and less voters involved.
Trust Neil on this one. Comment, comment, comment. The contestants appreciate it more than you know.
| 4 people marked this as a favorite. |
Having been in the Top 32/16 last year, I speak from experience when I say this: never did I feel like there wasn't enough focus on those of us in the competition or our entries. The revelation of the Top 100, the discussion of the personal keep lists, the Critique My Item thread...none of that ever felt like a distraction from my accomplishments or those of my competitors. While I appreciate the desire of the Paizo staff and judges to keep the focus of RPGSS on the Top 32, I doubt that the members of Top 32 worry about the diffusion of their limelight; they're too busy working on their designs for the next round (I know I was). We should give the Top 32 competitors the benefit of the doubt and assume that they will exercise sportsmanship and professionalism in encouraging initiatives on the part of Paizo to continue to engage those who didn't make the cut.
I have no personal stake in whether or not Paizo decides to reveal the Top 100; I'm objective about my skills as a designer and I know that I took several missteps in my Round 1 item this year. For me, the ranking of my item is immaterial if I'm not in the final Top 32. I would, however, urge the staff and judges not to withhold the list on the grounds that the emphasis should be kept on the Top 32. At the very least, I encourage the judges to give thought as to whether or not such actions truly serve the best interests of the top competitors because I don't think members of the Top 32 care. I know I didn't. :)
| 2 people marked this as a favorite. |
I'm going to disagree with James and Mikael a bit. Compared with 2012, I was really stunned at how few comments we got last year. I wrote about it during the contest and how I thought it really needed to have Round 2 back and have it be less crunch-intensive and again just a few weeks ago on Mikko's blog. Now, I didn't think it was because of the Critique thread (that's been around almost every year), but something's changed since public voting that's cut down on people commenting.
I really want people to focus on the Top 32 and I think Neil's right that they live (and die) for every comment they get. I'm planning to comment on every single entry in every round, barring something unexpected and I want to encourage as many people as possible to do that too. (I still want to hit all the items in the CMI thread too, but it's taking me a little longer.)
That said, I also think Garrett has a point. Getting into Superstar made me realize that I do have some ability to do this professionally. Without it, I never would have even been aware of the Compatible Products forum or known to look for open calls and find ways to get work. I think someone learning they've been in the Top 100 could have that same effect. Even in 2013, NOT getting into the Top 100 taught me something. I learned more about design and what didn't work by learning just how not popular my item was. I think there is some value in that.
Which, honestly, makes me want to not vote in Round 1, since my input seems to have little or no impact upon the Top 32 selection process. Sure, I helped weed out the bottom half of all entries, but a single gatekeeper skimming every entry could have done that just as efficiently behind the scenes.
By spending thousands of hours sorting items, the public is saying, "Hey, judges, here are these several dozen items most of us really want to see in the Top 32." And then the judges turn around and say, "I don't know why this item I like didn't get a lot of support from the public. I'm putting it in the Top 32, anyway."
<Snipping here>
On this topic, I personally disagree. I like the idea that the judges are, in fact, judges, and that there is a compromise between popularity and professional critique, for four reasons:
- Time: The voting process is long overall, but each item will receive around 30 seconds to maybe 2 minutes of consideration on average. That is not enough time to actually be comprehensive, and there is a lot to consider; in the CMI thread I think I averaged somewhere around 15-30 minutes an item (depending on item type), and even then I was hardly being that thorough. Once the sorting is done, the judges may still have a lot of items to look through, but they aren't working in the same rapid-fire fashion, and can go back to earlier items to reconsider.
- Priorities: Fundamentally, people who play games and people who make games professionally aren't going to have the same priorities in this kind of thing and both opinions matter: Most (or at least a statistically significant number of) voters will generally (longer ramble of this in spoiler-tags below) be looking for items that are new, cool, flavorful, have mechanics that make sense and they and their friends can have fun with at the table.
Paizo are essentially talent-scouting for freelance professional game designers, and as such are looking for something that displays that someone can create innovative but publishable items and has a head for mechanics and game balance, because they want to (potentially) hire them. Either in isolation is okay, but fundamentally less useful (for everyone) than a compromise between the two: What players want tempered by professional critique.
- Luck: As a disclaimer: I'm not a statistician, I don't know the numbers, nor the metrics for what-was-up/down-voted-against-what, so the following is just my opinion. I'm not sure how the voting system is coded, and whether it is truly random (though after seeing some items over a dozen times in under 400 votes, I grew dubious of the actual randomness involved... but I have that opinion of my blue d20 too). A random X-against-Y system with a large pool will generally get a very flat distribution, and placement on it is heavily based on what items it gets put up against.
Culls help reduce the randomness by removing the consistently downvoted items from recirculation, forcing differentiation between the middle-runners in particular, but they're best done in small bites and starting reasonably early due to loss of voters due to burnout. The later the cull, the less useful it is because less people tend to be voting, and the effect of the constantly-recirculated bottom tier items is hard to overcome (in theory everything should get about the same match-up on average, but for that with this size pool there needs to be a tonne of voting; I think we're all familiar with the fact that random chance really doesn't care about averages :P)
- Money: Paizo is paying people to run the contest, and will be paying the winners to produce publishable material for them. I think it fair to say that they have every right to keep a firm hand on the proceedings. I know I would if it was my company.
Most players I know and have talked to (who've played Pathfinder, obviously), there is an inherent trust in Paizo to handle the nitty gritty details of game balance, mechanical coherence and control of power creep. We pay for a Paizo product, we expect (and almost always get) a quality product. We don't expect or want to have to run quality control on the contents, modify items, forbid spells or do other heavy handed and involved adjustments; the product is expected to be usable off the shelf. And it generally is.
From what I've seen, this affects how a lot of people go through the RPGSS process with the items as well; Most of us focus on what it does, whether the mechanics are clear, and whether it is something we like and could see using at the table in some fashion (I don't, because I like numbers and I nitpick. I blame 13 years of engineering). And for the most part that's okay.
But Paizo are the guys having to do that work, so that we don't have to, and when looking to hire more content creators they're going to need some kind of influence on the selection process.
As far as posting the Top 100.... I personally don't think it's actually worthwhile. There was a contest: 800+ entered, only 32 could continue.
The judges and voters provide feedback-on-request in the CMI thread to help those of us who weren't in the 4% and... that's all that's needed. Seriously, how many job interviews have you been to where they said no, and then were willing to explain honestly and in detail why?
|
RPG Superstar 2014 Top 4, RPG Superstar 2012 Top 32
,
Dedicated Voter Season 6, Marathon Voter Season 7, Dedicated Voter Season 8, Star Voter Season 9
|
| 6 people marked this as a favorite. |
I think there are really good points on both sides of the discussion.
In my opinion, during the voting rounds, there should be more focus on the finalists and less focus on the rest of us. I want to encourage everyone to comment on the entries, it means a lot to the finalists.
However, I'd argue that between the rounds, any distractions (self-serving or not) that keep the audience's interest piqued are good for the contest. When people have cast their votes, written their comments on the entries, there really isn't much else they can do for the finalists at that point. Anything that encourages positive discussion is good for the contest as a whole.
My reasoning is that if the voters are more active and enthused between the rounds, they're also more active and enthused during the voting rounds.
All that said, I support Owen's decision not to release the top 100 list. I apologize if my little Detect Mojo contest has increased the demand for such a list and put more pressure on Owen. Considering how much energy and effort Owen has put into running and improving RPG Superstar, it really is unfair to demand more and more. He has my utmost respect for what he's done for the contest.
Regarding the suggestion that releasing the Top 100 is the best way to inspire people to keep trying...
RPGSS exists, at its heart, as an endeavor to provide Paizo with new talent who can come up with fresh, interesting, and sound ideas on short notice. Surely, then, the main reason that people will keep trying is that they madly want to work in this industry - or at the least be noticed - and they don't care how many times they are rejected, they will keep trying until they get there. Because Paizo is one of the biggest names in the RPG industry and the people who get far in RPGSS have gone on to do some amazing work.
Releasing the Top 100 doesn't really achieve much. It heartens/mollifies only 68 people out of the many hundreds that didn't succeed, never mind those who didn't even make the cull. For the rest of us it might serve some purpose to mine them for details as to what had potential... But I think there are better ways to design items than to attempt to figure out what people voted for in 2015 and create something accordingly. After all, voters will be swayed by any number of things – including movies, manga and video games that come out between now and 2016 - so what they voted for in 2015 might not be all that applicable next year.
Most competitions (art ones, to be honest) that I have been involved with would never release as much information as the RPGSS judges have already done. You would never get a personal critique from the judges. At best, you might get a letter thanking you for entering and informing you that you hadn't been selected. Most of the time, though, you don't get squat and you gotta deal with that.
But still we try, because we have that dream. We try because next year might be different. Because next year we might be the Superstars. By entering and losing, we have "almost made it". Knowing that we were in the Top 50 or Top 100 or Top 666 does not make us closer to having won. The only way to be closer to winning is to try again.
Based on the Critique My Item thread...
"Text wall warning"
This is a fun event that we all look forward to every year. It is an event to be supported and applauded for the invigorating gaming mojo effect it has on all contestants win or lose. Generally speaking, I defy you to find a similarly supportive and inclusive online community presence. Go the top 32!
I think the major problem with how people are perceiving RPGSS (and I am sad to see this) is that people...the contestants (see I did it myself!... I should say contributors)... forget that it is promoted as a casting call for talent. All that is happening is that Paizo is happy to let part of a process occur with our participation. Most casting calls are resolved behind closed doors. This is where I go a bit further than Raynulf - it suits Paizo to let everyone be involved, view and comment on the process... sort of like looking at a cut away section of an ant farm ...you know what I mean (roll a d20 to disbelieve in the idea of a contest!).
As long as everyone remembers all of this it is all fine and dandy but a lot of consternation seems to arise out of the process. Ultimately the process has little to do with what individuals want because, in the end, the judges will want to be seen to be selecting some talent, the system allows for preferential voting to a point to involve us in the process, the masses are engaged and happy and not rioting in the Forum and all is good. For goodness sakes what contest has the Judges voting in it! It is not a contest in the sense that most people suppose it to be.
There is nothing inherently wrong in all of this because its all pretty clearly set out but in the thrill of it all people sometimes forget. If, at the end, some worthy gets selected to have a chance to realise a dream then that is fine by me.
I am a firm believer in the RPGSS but I am not however, convinced about the "improvements" to the process. Having been intrigued by the idea of a cull I am now not a believer. As I have said (and noting the above and that I also like the whole idea of an observable event with a good community vibe) it alienates too many people and otherwise worthy items from the mind too early. I agree in not releasing a top 100.
Go back to the good old days...no cull, no top 100, only alts and top 32. Keep the mystery alive! The changes to the process are causing more comment than is desirable, keep the focus on the task that each round throws up and any yearly "tweak" to keep everyone honest. The convention is not to refer to items directly in the first round. The rule is not to refer to your own item. I'm inclined to the view that the punters should be able to comment as they will from the start. Let them defend their item. Let them express a view on other items by name rather than by doing so in all but name. Set some rules around how it should be done and have at it. It is currently set up as an election process after all so why not let the campaigners campaign. Let the contestants vote on what they consider their top 100 and top 32 to be and let the Judges go about their own assessment in their own way.
|
RPG Superstar Season 9 Top 4
,
Star Voter Season 7, Dedicated Voter Season 8, Star Voter Season 9
aka Snowblossom
|
I disagree rather strongly about whether someone "should" or "should not" need validation to keep at it. People need what they need, and it's different for everyone.
I absolutely agree, RD. The majority of people who work in this industry are creative, and creative-minded people tend to be more sensitive. In many cases, validation stirs inspiration and motivation. There should not be any shame in this. Everyone goes through moments of doubt and there is nothing wrong with that. While I do feel that resiliency is key to working in any creative field, that doesn't mean that validation isn't important to help strengthen that resolve.
This thread is full of sour grapes.
| 2 people marked this as a favorite. |
This thread is full of sour grapes.
That particular characterization bothered me when Neil said it and it bothers me to hear it mimicked... This is a civil discussion calling out the other side of a discussion as just being "sour grapes" is dismissive name calling.
Making sure every single person that requests feedback gets it is not "sour grapes." The former competitors tossing in their opinions on both sides of the discussion is not "sour grapes." This community is actually a very supportive and dare I say sportsmen like one. I don't see how a discussion on the merits and flaws of some of the changes this year is in any way against that spirit.
If you want to see something more akin to "sour grapes" go post something on most any gaming subreddit.
Owen if I somehow missed saying it before I'll say it now, thank you for carrying the torch of hosting the competition this year. I think I can speak for many of us in saying some of the curveballs have been challenging in an exciting way and I'd say the changes to the first 2 rounds of the competition have been a complete success.
I'd like to see a few of the traditions stay alive which for the past couple years included a little more communication in the blog announcements and teasers during the week we all collectively waited for the top 32. If it sometimes seems like I'm leading a mob, please understand we're more like a flash mob then an angry one.
I understand that taking over something as beloved as Superstar has got to be a hard job and the public facing and private facing comments I've made were in the spirit of providing feedback. With the written word we lose tone and body language so if it helps I suggest picturing and hearing everything I post in the voice an likeness of Puss-in-Boots... :P
| 3 people marked this as a favorite. |
Put me in the camp of people who would like to see the top-100 released.
I think it can be valuable to know how far off the mark your submission was.
Did it fail because the public didn't like it enough?
Or did it fail because after careful consideration from all 3 judges it was not deemed good enough?
If this information makes me self-serving, and not superstar-caliber in some people's eyes, then so be it.
didnt read all the wall of texts
In any foot race the competitors always know their position as they can see the completion and their relative location to one another. After the race the competitor can then judge themselves, by their individual race times, to the finalists into how they can improve.
Since this competition is done in groupings. Maybe instead of releasing individual rankings, just release the consolatory groupings of the individuals. As that may motivate an individual who is thinking about not competing in further contests, to give it another try.
Since we know the top 33, it would be nice to know who was in the runner up 66 (33-99) , and maybe even the dont give up 132 grouping (100-232)
RPG Superstar 2012 Top 32
,
Champion Voter Season 6, Champion Voter Season 7, Champion Voter Season 8, Champion Voter Season 9
|
| 6 people marked this as a favorite. |
I am a numbers guy. While I really don't care what the Top 100 items were, I would like to see a post about the voting numbers with the following things:
- Number of votes cast (rounded to nearest 10k).
- Number of voter tags broken down by category.
- Total number of votes cast by each category (possible?).
- I would like to know after which item number there was a clear drop in votes per item. Like Top 89 of first public voting year.
- While it would be nice to know the total number of submissions, I know that number is never released.
- Percentage of submissions disqualified (curiosity).
- Did any items not get voted up (without listing name)?
Yeah I really missed that post this year...
...XXXHey, Lucus...great points. Here's a short response to some things you brought up, which I think you should consider:
** spoiler omitted **
This mostly to Neil, but everyone's free to hear it. I'll spoil it so it keeps the thread manageable.
First up - Thanks for your quick response Neil!
I said I was "putting it off", not delaying it indefinitely. I'm, for instance, starting my thread off today with comments about the 4 Alternates, because from what I can tell, at least at this stage, none of the Top 32 are DQ'd and thus no alternate is getting tapped.
<side note: starting off as in, I took a purposeful break in time to let the maps have some time before I start filling the Gen Disc. forum with my thread again>
I have limited time, like everyone, to do it all --- and at least for the next 5-6 days, the Top 32 are vested to come back as their maps are still in the balance. I've even considered, like I'm doing for the 4 Alternates that are effectively "out" of the competition <not yet, but its still probable> to wait to comment on the Top 32's items till they are out (so I'd wait to comment on the Top 16s, but would comment on the 'Bottom 16' so to speak, on Feb 3rd. etc.) That may not be able to help the Top crew as they move forward, but if they're Superstars, they don't need the helping hand ;-P
I doubt I'll do that, I'm making solid progress on the Official Thread and when I get caught up, I'll likely do all the Top 32 in some fashion.
But that all said, I still feel, looking over the comments that most of what I'd say has been said, and while I'm sure they'd love another pat on the back or a suggestion about their item, I'd rather they stay focused on their maps and the comments there.
Considering my comments above, I think I will try to make time to add comments and my questions to all of the Top 32 entries, not just my personal Top 8+.
You've sold me on the fact that I should ask those questions, and I actually might have a mildly "interesting" means of approaching it... "How many different interpretations of this map can I make?" which may sound odd here, but I hope if I do it, it will be insightful to the process of map-making, critique, etc.
Regardless, thanks again Neil for responding.
And regardless of how this conversation continues, I just love that it's able to happen in a way that we can all participate in. Other such competitions just don't have this level of engagement (I'm meaning competitions like Country-of-Choice Idol, or XYZ's Got Skillz).
| 1 person marked this as a favorite. |
Hey, Lucus. As long as you engage the Top 32 competitors (and their submissions) with feedback of any kind, I'm good. Sounds like you've got a decent plan for doing so.
This thread is full of sour grapes.That particular characterization bothered me when Neil said it and it bothers me to hear it mimicked... This is a civil discussion calling out the other side of a discussion as just being "sour grapes" is dismissive name calling.
Hey, Frank. I don't mean to belabor it or anything. I've read your private message, and you've already edited your original rant. But I do want to indicate that I standby that characterization of "sour grapes" because that's how you came across with some of your word choices and phrasing. We all know it's difficult to interpret the emotional content in someone's written words via the internet. But, there are certain words which can indicate a level of "heat" that the original author might have never intended, but they're taken that way anyway.
With regards to your original, pre-edited post, there were some instances of that, which, if you go back and re-read your original words should give you an idea of why folks took it that way. And, by that, I mean more than just me, as others have indicated it, too...hence, the mimicking you took exception to. I'm not really sure I'd call it mimicking, though. It seems to be more of a shared opinion from multiple people.
Regardless, let's clear the air on it...and, as you indicated privately, agree to disagree.
Several people have chimed in saying they'd favor a release of the Top 100. There's precedent for it based on prior years of the competition. I'm sure it helped motivate a few who found themselves on that list. Personally, I also think it does a disservice to the Top 32, but not all of them feel that way, as evidenced here by at least one member of this year's crop.
So, in the long run, it's Paizo's decision on what they choose to do. If they want to maximize the level of engagement among the voters, I guess they can keep expanding on the transparency by posting it up. And, if they want to foster more focus on the actual competitors and ensure more engagement in their submission feedback threads, I think they'd be wise to withhold it. Why? Because I think a lot of people stick around for the "Critique My Item" thread and the Top 100 posting, but, as soon as they get that insight, they tune out the rest of the contest. And that's because they're not as interested in supporting the actual competitors as they are in extracting something for themselves. I know that sounds harsh, but also believe it to be true. Not for everyone, of course. Many do stick around after engaging those opportunities for personal feedback. But others take their submission and go home as soon as they get their personal insights.
But that's just my two cents,
--Neil
RPG Superstar 2015 Top 32
,
Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Star Voter Season 9
aka mamaursula
|
| 3 people marked this as a favorite. |
...A longer explanation of what I mean, with special thanks to Owen, Neil, and Garrett...Hey, Lucus...great points. Here's a short response to some things you brought up, which I think you should consider:
** spoiler not omitted **
So, if you enjoy all that commentary you get from people in the "Critique My Item" thread, consider passing on that same experience to the Top 32 by giving that to them, as well. Every bit of feedback is absorbed by these designers. And, quite honestly, they need it to grow over the course of the competition. So, don't discount what your voice can add to the comments on their submission threads. I assure you, as a former-competitor, we tremendously value it.
The site will not let me like this comment 100 times, so I have to comment.
Much like this year, in the past I have taken to radio silence because my husband competed, but I could give positive encouragement. It is incredibly grueling to be a contestant and there are times when you just want to lay down and quit because you don't know what you don't know. Please, be upbeat, positive and cheerlead for ALL the contestants, we need that from our audience, you guys. Every year 32 real live people compete to bring the very best we have in our hearts, minds and game designing prowess to the voting polls in hopes of giving you a fabulous gaming experience. And it's something we've never done before, this is the Olympics of game design, amateurs competing for the gold. Cheer for us. Cheer your hearts out for us, because we need it and we want to please you and ourselves.
|
RPG Superstar 2013
,
Dedicated Voter Season 6, Dedicated Voter Season 7, Dedicated Voter Season 8, Star Voter Season 9
aka Steven T. Helt
|
| 1 person marked this as a favorite. |
Speaking from personal experience, be careful. There's a thin line between firm criticism and perception of sour grapes. Even if you feel like you have a pretty strong point, people can misunderstand, or (again, speaking from personal experience) you can say the wrong thing.
We can all agree the contest is a great thing, and we can all agree we want the contest to be the best talent search it can be. Maybe couch every suggestion in terms like "this is something Paizo should think about for next year". Don't say "this was a bad move on your part".
I fell like there's no harm releasing the Top 100 at some point, but I think it should be dropped in the official critique thread AFTER the top 16 are announced. The amount of time devoted to the contestants necessarily decreases with the number of contestants, so jumpstarting the critique thread again AFTER there's been ample time to celebrate the top 32 and vote on their entries seems smart.
Credit Paizo for having the most transparent process and giving new designers opportunities no one else does.
As far as Mark's feedback and selection process, make sure you guys understand..there are hundreds of items in round one. If one or two judges doesn't get started offering feedback IMMEDIATELY as each item comes into the system, there is no way the first round judges will be able to offer a thorough critique of 50-150 items in order to have the selection process completed by Monday night and ready for the reveal Tuesday afternoon. Having seen the other side of the process this year, I have little doubt judges in previous contests have evaluated and judged entries long before a top 100 was generated. For most years of this contest, the judges alone picked the top 32 so you know they all behaved exactly as Mark did. It's criticizing that process in the face of common sense that drives people to call sour grapes.
While we're talking feedback about the process, I just want to say, I think the contest is better when the same judges judge each round. Maybe that's just to the credit of the specific judges in those days, but the ability to get to know a design luminary and learn their tastes, experiences, etc was a big draw for me. I get the appeal of using the different judges, and I think those different judges are doing a good job. But having a new batch of pretty good judges every round isn't the same as having SKR, Wolfgang, Clark, Owen, Neil etc every single round throughout the contest.
And that my two cents.
(here's your quarter, Neil.)
| 4 people marked this as a favorite. |
Eric, I think that the critique my item thread will allow each individual designer to do just that in a way that is even more directly useful to them. Ask for your judge feedback, and I'll supply it. The more and longer, more comprehensive negative bullet points on the judge feedback, the more likely you failed to impress the judges (and you get to know why too, in detail). The fewer the comments (and especially if they are more generally neutral or positive, but either way), the more likely that it was the voters you failed to impress. You don't get to know why, as well, in that case, except as much as the voters help you by critiquing your item too.
| 2 people marked this as a favorite. |
I think we all need to step back for a moment and remember, at the end of the day this is Paizo's contest to find talent to work for them.
SKR had a post, I believe the first year in voting, that was in effect saying it shouldn't matter if you item 33 or item 333. You didn't make Top 32, and that means you have more homework to do, more practice to do, and more writing to do.
I can see it from both sides, my human side wants to be validated, to know I made it at least to the point where judges seriously considered my item. I also want to know if I was in the Top 100 (or whatever) to be able to know that I am on the right track in my writing and development.
I also understand not posting, that means less feelings hurt, less chance of rage posts, and allows me see who is going to be professional enough to just go an practice more and come back next time. There is a risk that the focus is shifted away from the contest and is put on the folks who are no longer in it (although I believe most of us can multi-task). It is right that those who are actually still in the contest get focused on, critiqued, and made better for the subsequent rounds so that Paizo gets the best, and we as the fans get the best material written by that best.
My goal is to be unable to enter RPGSS every year, because I've been published on hardcovers enough. It has been said before, this is not the only way to enter the hallowed ranks of Paizo freelancers...it is one way. I believe it is just as impressive when there is a freelancer who worked their way to join the Paizo ranks, and then continually produce. Not that there is anything wrong with doing well in RPGSS and producing a couple of things, but I can see how burnout could set in from the competition's stress and then tight deadlines from there. That's also not to say that those who do well in RPGSS fall into that trap. It all comes down to say that this really isn't the only way to write for Paizo professionally.
I am fine with Paizo's decision not to release the Top X list. I sincerely respect that the desire to see that list is being considered for next year's contest. I definitely don't think it is worth getting too wrapped up in.
Eric, I think that the critique my item thread will allow each individual designer to do just that in a way that is even more directly useful to them. Ask for your judge feedback, and I'll supply it. The more and longer, more comprehensive negative bullet points on the judge feedback, the more likely you failed to impress the judges (and you get to know why too, in detail). The fewer the comments (and especially if they are more generally neutral or positive, but either way), the more likely that it was the voters you failed to impress. You don't get to know why, as well, in that case, except as much as the voters help you by critiquing your item too.This is a very useful insight into the judges' commentary.
Thanks for sharing this, Mark.
No problem! It's not a guarantee; sometimes you will see a longer set of negative commentary even on one that the voters also didn't like, simply because there was something so negative that it stood out, in which case it failed to get past the voters but would have very likely also failed to get past the judges anyway. In any case, I personally find this version to be even more useful to more designers than a top list would be (particularly to designers that would be just below the cut-off of the revelation, say #101 if it was Top 100, who might otherwise be more discouraged).
RPG Superstar 2012 Top 16
,
Star Voter Season 6, Dedicated Voter Season 7, Dedicated Voter Season 8
aka FaxCelestis
|
See, though, the top 100 list does give feedback. If you made the top 100 but not the top 32, you know you impressed the public but not the judges. If you didn't make the top 100, you didn't impress the public. That means you have different things you need to specifically work on.