Chief Technical Officer
|
Add a way to flag R1 items that break the rules -YES!-
We did actually have a way to do that at the start of the voting period. It didn't take long for voters to help us identify most of the problematic entries, and then it became nothing but false positives (things that people *thought* broke the rules, but didn't), so we turned it off.
That worked well, and we will likely do it again.
What people were asking for was the ability to tell us *why* they thought it broke the rules when they flag it. I'm personally not sure that it's worth the programming time it would take to implement leaving a message as part of the flagging system (and also to implement allowing the judges to view those messages) since—if an entry *actually* breaks the rules—the judges are almost always able to figure out the reason once their attention has been drawn to it.
The only notable exception to that is plagiarism. However, this year, a lot of people flagged entires that they *thought* were plagiarized that were not. For example, a wondrous item that's a big blue box which is bigger on the inside and travels in space and time is not actually plagiarism; it's unoriginal, and will not win a lot of votes from the community because our community is smarter than that... but it's not plagiarism.
RPG Superstar 2012 Top 32
,
Champion Voter Season 6, Champion Voter Season 7, Champion Voter Season 8, Champion Voter Season 9
|
Add a way to flag R1 items that break the rules -YES!-What people were asking for was the ability to tell us *why* they thought it broke the rules when they flag it. I'm personally not sure that it's worth the programming time it would take to implement leaving a message as part of the flagging system (and also to implement allowing the judges to view those messages) since—if an entry *actually* breaks the rules—the judges are almost always able to figure out the reason once their attention has been drawn to it.
A second dropdown menu with 3-4 obvious rule-breakers and then "Other"?
| 1 person marked this as a favorite. |
Or more simply a check box on the item such that this happens
a) voter says "I think it breaks the rules"
b) flagged, gets checked by Judge, who kills or saves it... saving it then...
c) sets a "has been checked and is ok flag" which disables the "I think it breaks the rules" for that item going forwards.
... that way, an item can only be flagged for checking just the once?
|
Chief Technical Officer
|
A second dropdown menu with 3-4 obvious rule-breakers and then "Other"?
If it obviously breaks the rules, it's going to be obvious to the judges, and if it's not obvious, "other" doesn't really tell us anything useful.
It's still spending valuable programmer time so you can give us information we can quickly and easily figure out for ourselves.
For myself, I'd kind of like a round which was a bit more rules-lite. A concept round.
We've had several people suggest this.
Compared to game rules, flavor is easy. Like, ten times as easy. There are many, many freelancers out there who are great at flavor but weak on the Pathfinder rules (for example, people who know 1E, 2E, 4E, GURPS, or Storyteller really well but not PFRPG). I can't use those people because they write nice flavor and I have to scrap and redesign their rules, which means I'm doing their job for them.
It's RPG Superstar--Roleplaying Game Superstar--not Flavor Superstar. There are fiction contests all over the web for rules-light material and concept material. This is a competition for game designers.
But what about something like organizations from 2012?
Was that still too rules-light to be helpful for you? If not, I think that's the type of thing that could be attractive vs. what I felt like was much rules-heavier archetype rounds in 2013 and 2011. I wonder if there's some happy medium in that area...
(If it was eventually too rules-light -- an attempt that didn't work for Paizo -- then that's less helpful, obviously.)
But what about something like organizations from 2012?
Was that still too rules-light to be helpful for you? If not, I think that's the type of thing that could be attractive vs. what I felt like was much rules-heavier archetype rounds in 2013 and 2011. I wonder if there's some happy medium in that area...
(If it was eventually too rules-light -- an attempt that didn't work for Paizo -- then that's less helpful, obviously.)
That's kind of what I was thinking. Not rules free, but a round where flavour is more important / more of a focus than the rules themselves?
|
RPG Superstar 2012 Top 32
,
Champion Voter Season 6, Champion Voter Season 7, Champion Voter Season 8, Champion Voter Season 9
|
But what about something like organizations from 2012?
Was that still too rules-light to be helpful for you? If not, I think that's the type of thing that could be attractive vs. what I felt like was much rules-heavier archetype rounds in 2013 and 2011. I wonder if there's some happy medium in that area...
(If it was eventually too rules-light -- an attempt that didn't work for Paizo -- then that's less helpful, obviously.)
I liked the organization round, even though it was where I left the contest. You could add a bit of rules; creating new traits, feats, spells, or another bit of rules crunch to define the organization. Too bad Paizo hasn't yet created rules for creating organizations similar to the GMG settlement rules... (nudge, nudge)
RPG Superstar 2013
,
Dedicated Voter Season 6, Dedicated Voter Season 7, Dedicated Voter Season 8, Star Voter Season 9
aka Steven T. Helt
|
| 1 person marked this as a favorite. |
An organization round with a trait and a feat is a great idea for a second or third round. 400 - 500 words total, some design you don't see in Superstar, and some Golarion development that makes players and GMs salivate.
I think the Org round from that year was an "easy mode" challenge and didn't really help us find a better game designer--at least, not in the same way that an archetype or stat block round does.
|
RPG Superstar 2013
,
Dedicated Voter Season 6, Dedicated Voter Season 7, Dedicated Voter Season 8, Star Voter Season 9
aka Steven T. Helt
|
Agreed in principle. But then (at least to me), "how can we make this better" includes amping audience participation and excitement. I think an organization round with more crunch, and/or a celebrity GM approach to playtests in the encounter round would do those things. I acknowledge every decision should be about finding better designers, but also about landing a larger audience and more entrants. IMHO.
I personally like the archetype round, I feel like those who will bring the fluff will do so along with the rules-fu. Those who will only bring one or the other will do just that. What else can match that?
Not sure, maybe an organization with traits and feats rules. Maybe an NPC and story hook. Maybe a new type of environment?
I like the idea of organization with a feat and/or trait, which I feel could be a little less rules-intensive (and thus maybe keeping the audience more engaged?) but hopefully still enough to be useful to Sean and Co.
Do settlements have enough crunch that they could work? I think that could also be a lot of fun (especially if you open it up to Distant Worlds).
One thing that might keep people around longer is to open the 'critique my item thread' later. For example opening it after voting closed but before winners were announced on a later round, (though there will be the threat of shadowing the Top 16/8/4). With the longest break before R5 it might keep board activity up until the final four need a vote. I think it will reduce the number of people participating in that thread which is sad and not sad at the same time :)
I'd like to hear your thoughts on how we can improve the competition and the process for the next RPG Superstar competition. Ideas that are already floating about are:
* Shorten the voting period for R1. We went with the 3 weeks because we weren't sure how much voting input we were going to get and wanted to leave enough time to sort all the items, but clearly that wasn't a problem. Anywhere we can reduced the "lag time" in the competition is good (as it is, the competition is almost four months long, and it's easy for the public to lose interest over that timespan).
* Implement the R1 culling earlier to avoid so much exposure to the really poor items. And perhaps add a second culling.
* Add a way to flag R1 items that break the rules (word count, copyright violations, and so on) so you don't have to send a private message about it.
* Add a way for voters to add a short comment to an R1 item that remains hidden until the end of R1, at which time the author can see all the comments.
* Drop the playtest step from R4. (Adding a week to the process late in the competition contributes to people losing interest in/track of the competition, and we don't get many people who post their playtest experiences, so it's not a big benefit that offsets the loss-cost.)
No guarantees on any of the above, like I said, they're just ideas, and I'd like to hear about any other ways you think we can improve the competition.
This is the first time I've seen this thread, and I couldn't be bothered to read all the way through it. Sorry if I'm repeating something that others have posted before.
I think dropping the playtest step from R4 is the best option listed; you Paizo staffers know how to spot a functional scenario, and we uneducated fans know how to find the most awesome of the choices. Those are the two sides of the equation, and with such a tight schedule play testing becomes a bit redundant.
Outside of the choices offered, I think R1 is giving you guys the most trouble. It felt like a disaster this year, if only because I'm not willing to weed through anyone's slush pile unless they pay me for doing so. This probably isn't very helpful, but a special forum title isn't enough reason to sort through all of round one, forever and ever. If, on the other hand, you would offer a recast of the Varian Jegarre Reaper Mini (in resin or whatever) I would probably vote on 5 to 6,000 unique entries.
I hope that all made sense.
|
Chief Technical Officer
|
A second dropdown menu with 3-4 obvious rule-breakers and then "Other"?If it obviously breaks the rules, it's going to be obvious to the judges, and if it's not obvious, "other" doesn't really tell us anything useful.
It's still spending valuable programmer time so you can give us information we can quickly and easily figure out for ourselves.
If someone particularly feels the need to explain why they think something is plagiarism, surely they can just PM the judges, given the Paizo boards actually have a message facility these days?
Yep! That's the way to go.
|
Chief Technical Officer
|
Outside of the choices offered, I think R1 is giving you guys the most trouble...
Getting people to vote in round 1 was not a problem *at all*. We were very pleased with the community response. We got more data than we really needed, so I think shortening the window is the main thing to do there.
awesome stuff here
1. The voting period was very long. It begun to feel like more of a chore than something fun to do.
2. I would also advise the cull to happen earlier. A second cull would be good too. After the first, it did remove a lot of items I hoped to never see again, but there were still several I felt didn't make the cut and should have been cut.
3. Flagging items would be great, whether I can add comments or not, I'm on the fence honestly. If I have a solid selection of reasons, I'll use them. If it's really off the wall, a 'Breaks Other' will be fine.
4. The play test is a nice to have feature. But really, If i want to play test something, I'd do it without a time frame for it.
5. This one would take more back end/heavy lifting but get a better sort of the items. There were several items in the top 32 I never saw in the voting period. But there were also several items I kept being asked to vote on, and vote on again and again. Sometimes it was the same items, just left/right reversed. Became frustrating to keeping seeing the same items over and over again and more so when I'd see the same pairings.
| 3 people marked this as a favorite. |
Obviously, all judging is subjective, but it's a matter of public perception more than anything else.
You'd never see a jury seated with no women on it. "American Idol" and similar shows all have at least one woman on the judging panel. Justice is supposed to be blind and impartial, but it's generally accepted that it's a good thing to have women and minorities on the Supreme Court. No establishment who expected to be seen as having a firm commitment to diversity would have its hiring or admissions offices made up of nothing but white males, even if they're the most fair and impartial guys in the world. It just looks bad.
I would predict that if Paizo put a female judge on the R1 panel next year and promoted it, they would see an uptick in submissions from females who felt more welcome because there was someone "like them" on the panel. It would demonstrate Paizo's commitment to increasing female presence in the hobby more than any podcast or marketing campaign or similar gimmick.
I just wanted to say a big "thank you" to Paizo and the individual organizers who clearly took this point to heart. I immediately noticed that every single round's judging panel has a woman on it. This hasn't gone unnoticed, nor unappreciated.
Seconded! (and Thirded from the wife)
| 1 person marked this as a favorite. |
I’ll treat it as a nice social experiment, if it draws in more submissions, great.
But we can only judge it a success if a woman makes top 32. Given that making top 32 is not easy……. I’ll appreciate it as a nice gesture but it won’t guarantee anything, which is as it should be. Two items enter and only the best advances.
| 1 person marked this as a favorite. |
I couldn't help but think that the panels could use EVEN MORE diversity, because while this is a great step, I'm still seeing a bunch of white guys and token white females (only 20% of each round). I do understand that it's hard enough finding qualified judges as is, so Paizo would be hard-pressed to deliver on a rainbow coalition of them.
In any case. Good job Paizo. It is appreciated. :)
| 1 person marked this as a favorite. |
Given that the entire WORLD gets to select top 64, I really don't think it is an issue.
Anyway, I am feeling a little bummed that this is moving into PC (political correctness) territory, I think it detracts from the fun.
We (the public) in all our diversity decide who advances each round, it is good to remember that.
Eh, as I said, they can't account for demographic disparities, and SHOULD pick from among the most qualified and interesting judges. But if they end up finding diverse potential judges. they may as well go for it. I feel like it doesn't hurt anything, but it definitely helps something.
The real diversity here is the judges' creative backgrounds though. Cartographers, writers, artists? That's exciting!
The real diversity here is the judges' creative backgrounds though. Cartographers, writers, artists? That's exciting!
Yup, that is most interesting.
Anyway, the contest for me is all about the ability to be AWESOME.
If a writer manages to do that everytime then that writer deserves, nay earns, RPG Superstardom.
The real diversity here is the judges' creative backgrounds though. Cartographers, writers, artists? That's exciting!
I'd been unaware of John Compton's pre-Paizo resume, but...
"John Compton, once an archaeologist and a teacher..."
...Whoah. INDIANA JONES IS A JUDGE.
Lol, better not make anything snake related then :P
| 3 people marked this as a favorite. |
One of the things I like best about RPG Superstar is that it is a contest that rewards talent, dedication and hard work above all else. Most of us would be hard pressed to put a face on the names of most finalists, or even the winners. Contestants are judged on the strength of their submission and their ability to engage with the voting public in a professional and curtious manner, regardless of their gender or race.
That being said, the importance of rolemodels and inspirational figures can't be understated, and I think it's great that female gamers are also represented among the judges. Especially because each of them have clearly earned their spot there. A copy of Crystal Fraiser's The Harrowing sits prominently on my gaming shelf, and I'm thrilled at the thought of potentially seing her comments to an adventure pitch i've made, just to name one example.
Jacob, thanks for saying it better than I can