| 2 people marked this as a favorite. |
I'd like to hear your thoughts on how we can improve the competition and the process for the next RPG Superstar competition. Ideas that are already floating about are:
* Shorten the voting period for R1. We went with the 3 weeks because we weren't sure how much voting input we were going to get and wanted to leave enough time to sort all the items, but clearly that wasn't a problem. Anywhere we can reduced the "lag time" in the competition is good (as it is, the competition is almost four months long, and it's easy for the public to lose interest over that timespan).
* Implement the R1 culling earlier to avoid so much exposure to the really poor items. And perhaps add a second culling.
* Add a way to flag R1 items that break the rules (word count, copyright violations, and so on) so you don't have to send a private message about it.
* Add a way for voters to add a short comment to an R1 item that remains hidden until the end of R1, at which time the author can see all the comments.
* Drop the playtest step from R4. (Adding a week to the process late in the competition contributes to people losing interest in/track of the competition, and we don't get many people who post their playtest experiences, so it's not a big benefit that offsets the loss-cost.)
No guarantees on any of the above, like I said, they're just ideas, and I'd like to hear about any other ways you think we can improve the competition.
This is wishing for the moon but:
* A way for the system to track items you've seen and shorten the time before the Submit button is available.
* Possibly just shorten timespan until Submit button is available for all item pairings in R1.
* This would probably alter the spirit of the contest but, a day or two period where the public can pose questions to the designer in R2 and beyond and the designer is allowed to respond.
***EDIT***
* Possibly base the timespan until submit button is available on how many you have already cast. Shorten it as the voter casts so many X votes.
| 2 people marked this as a favorite. |
I'd like to hear your thoughts on how we can improve the competition and the process for the next RPG Superstar competition. Ideas that are already floating about are:
* Shorten the voting period for R1. We went with the 3 weeks because we weren't sure how much voting input we were going to get and wanted to leave enough time to sort all the items, but clearly that wasn't a problem. Anywhere we can reduced the "lag time" in the competition is good (as it is, the competition is almost four months long, and it's easy for the public to lose interest over that timespan).
* Implement the R1 culling earlier to avoid so much exposure to the really poor items. And perhaps add a second culling.
* Add a way to flag R1 items that break the rules (word count, copyright violations, and so on) so you don't have to send a private message about it.
* Add a way for voters to add a short comment to an R1 item that remains hidden until the end of R1, at which time the author can see all the comments.
* Drop the playtest step from R4. (Adding a week to the process late in the competition contributes to people losing interest in/track of the competition, and we don't get many people who post their playtest experiences, so it's not a big benefit that offsets the loss-cost.)
No guarantees on any of the above, like I said, they're just ideas, and I'd like to hear about any other ways you think we can improve the competition.
All of the above, yes please!
1) When submitting your R1 item have a checkbox enables the item to autopost in the submit my item thread and have that thread locked. A separate thread is started to leave feedback on the items making that process less of a clusterf**k.
2) Maybe some sort of tags for people who write tons of reviews. MAybe at the end the judges pick s few posters as superstar citizens for promoting the competition or in general demonstrating good sportsman behavior (Anthony Adams for example, but also theheadcase, ect.)
3) Late round voter raffles for Paizo store credit might help keep some folk interested in later rounds.
4) Throw up some polls on various topics maybe come up with 3-5 twists and let us vote on the twists each round
5) No offense but this year the judges seem a little less involved with the discussion threads (as compared to looking at previous years forums), change that and you'll also get more engagement.
6) Remove the gag order, let the competitors engage us and us engage them. The worry that it might influence the competition is really moot. The worst that could happen is someone talented loses votes because they act like a jerk, do you really want to work with someone that unlikable anyway?
|
Good ideas, SKR!
I was definitely one of those people who had trouble maintaining my interest over the whole contest.
*I really like the crowd-voting first round process, but I agree that it was kinda too long. Good idea about culling earlier/more often, and having better reporting features.
*I was a little surprised that we only got 8 votes in round 2, instead of 16. Was there a statistical reason why this had to be the case?
*Shorten up the voting periods for the later rounds as much as you can.
*Make sure that the rules and expectations for Round 4 are a little clearer. It seemed to me that the "feature a creature" requirement in particular threw some competitors off. Also, the difference between "creating a location" and "creating an encounter" can be a little vague the way the rules are currently written. It seemed to me that some competitors might have been unclear as to what you were expecting.
RPG Superstar 2012 Top 32
,
Marathon Voter Season 6, Dedicated Voter Season 7, Dedicated Voter Season 8
aka Jiggy
|
| 2 people marked this as a favorite. |
I fear I'll sound whiney, but the R1 judge feedback that I saw people getting last year was sorely missed (by me, at least) this year. Several voters gave me feedback, but the vast majority of that feedback came down to either "You should have defined X" (when X was a Core-defined term) and/or "Y makes it horribly overpowered" (when the math proves Y just makes it less underpowered). One voter even changed his mind once he was up to speed on how things actually worked, but of course by then it was too late.
Ultimately, I was left with little to no feedback that will actually help me make a better R1 entry in the future, and therefore eagerly awaited judge commentary so that I'd have something to work with. (But I never got it.)
So yeah, if there's any possibility of bringing back judge feedback, that would be fantastic.
I fear I'll sound whiney, but the R1 judge feedback that I saw people getting last year was sorely missed (by me, at least) this year. Several voters gave me feedback, but the vast majority of that feedback came down to either "You should have defined X" (when X was a Core-defined term) and/or "Y makes it horribly overpowered" (when the math proves Y just makes it less underpowered). One voter even changed his mind once he was up to speed on how things actually worked, but of course by then it was too late.
Ultimately, I was left with little to no feedback that will actually help me make a better R1 entry in the future, and therefore eagerly awaited judge commentary so that I'd have something to work with. (But I never got it.)
So yeah, if there's any possibility of bringing back judge feedback, that would be fantastic.
See number 1 on my list. I think if the thread is less of a nightmare to keep up with and more controlled, its easier to add feedback. IF itas that engaging it can even be a reward for folks that continue to participate in the community around the contest.
4) Throw up some polls on various topics maybe come up with 3-5 twists and let us vote on the twists each round
I second this!!
| 1 person marked this as a favorite. |
* Auto entry for previous contestants? ; )
* Two contests a year?
* Field of 64?
I suppose I'm mostly joking on those (though as a huge fan of Superstar, I would totally love any of them) as I don't think they're very realistic changes without vastly increasing your workload.
In terms of the ideas you've listed:
* I didn't mind the length of the voting -- I was motivated to get to marathon status -- though a second cull would certainly have been nice.
* I've tried playtesting Round 4 this year and last and agree that I'm not sure it's that helpful. I was playing via PBP, but I couldn't actually get to them all and found myself tempted to run ones I was already inclined to root for. I wouldn't mind seeing that extra week go, or perhaps be used for something else...
To wit, I wonder whether a sixth challenge would help increase interest, though that might require actually going up to 64 contestants (or going from 32 to 24 to 16...). I think Round 4 gets really complex -- and while I can see its use to show more overall design skills -- I wonder if it would be useful to have a round to create a location before getting to what is now Round 4 with the whole encounter. As an observer, I find it difficult to decide what factor will ultimately determine my vote -- I tend to gravitate more to looking at the location itself but there are other factors, especially when there was also a trap component this year.
A few other thoughts:
* I love knowing the twists early. That really was fun for me to play along with as a spectator in R2 (and I think probably would have been hugely helpful as a contestant last year). I think especially with R2, we saw a lot of people who didn't ultimately get into the contest participating in that. I think that also goes to interest -- it starts dropping off a bit as people aren't participating themselves in designing. Letting us participate as long as we can is going to add to our overall interest level, I think.
* I really like the twists involving previous rounds. I'm glad that was reintroduced this past year after going away in 2012.
* I like Solspiral's idea of letting us vote on the twists (though of course that could potentially conflict my points above). I think that could definitely increase participation through the contest.
* I also agree with Trinite that limited number of votes in Round 2 was frustrating. Letting me vote more makes me more invested in more players, I think, and thus more interested in following along.
* I also have to agree with Jacob 1 in that I do miss some of the judging feedback. I realize it's a huge effort on your ends, so it does feel selfish to ask, but it was something I missed. I understand why my item didn't get votes from the public, but I'd still love to hear professional designers' feedback. Paizo's so good about interacting with fans -- witness this very thread -- so it was tough having that curtailed a bit this year.
* I loved the pitch sheet preventing module concept duplication that was introduced this year. Please keep that for future competitions.
* Force Neil to come back. : )
7) Round winners get published. You clearly win the round its going into something.
8) Round 4.5 the wild card round. The 24 + alternates that didn't make it to round 4 get one last design challenge to come back. Only 1 can win and the voters must pick.
9) Partner with 3PPs that also want to run contest and cross promote one another and time it so that there's always a contest out there, this way your create a community that stays active year round.
This might also be too hard or not worth the effort but:
* On the R1 item set presentation algorithm, find a way to ensure that reverse pairings are not counted as "separate" pairs. For example Item A vs Item B and Item B vs Item A should not be considered 2 separate pairings.
* On the Monster Design round, could it be possible to give the designers the fabled number crunch spreadsheet to weed out those types of issues? Maybe you don't want them to have that to test their capabilities in that respect...among other reasons.
* This is divisive...but MYTHIC round next year!!
* This is divisive...but MYTHIC round next year!!
light the torches and assemble the mob!
I'll make it even worse then. Next year I want a cursed item crafting round!
That would be awesome
RPG Superstar 2013
,
Dedicated Voter Season 6, Dedicated Voter Season 7, Dedicated Voter Season 8, Star Voter Season 9
aka Steven T. Helt
|
| 1 person marked this as a favorite. |
Any rules set for Pathfinder should be fair game. Whether the challenge is Mythis or psionic or anything else. Superstars are tapped to provide material for future supplements. You don't want a Superstar that can't/won't design when you ask them for a product.
I think every year should have a villain round. Adventures need memorable villains, and early years of Superstar really separated the cloud giants from the hill giants. Still giants, of course. But designers who grock villains tell good stories.
I dig the short feedback button, I dig shorter voting, and I think a second call is neat. Remember that the public voting process created a very identifiable Top 89, so it was a great system first pass.
I think an alternative to dropping the playtest round is to improve and incentivize it. Suppose Paizo asked eight well-regarded GMs to playtest one encounter each. With a little notice, they can play at home and observers could Skype in. Or PFS organizers could hold them at LGSs, or the RPGSS Eight could run pbps with voting public players. There a lot of ways to add buzz to the playtest round without spending much in the way of resources.
I have more ideas on the playtest part, but really, I think RPG SS worked out GREAT this year! :b
* This is divisive...but MYTHIC round next year!!
Actually, I like this one. It would be a real test of a contestant's skill with bringing something of mythic power without expanding the power level beyond all reason. A great test of mechanics and cool-fu.
| 1 person marked this as a favorite. |
I think an alternative to dropping the playtest round is to improve and incentivize it. Suppose Paizo asked eight well-regarded GMs to playtest one encounter each. With a little notice, they can play at home and observers could Skype in. Or PFS organizers could hold them at LGSs, or the RPGSS Eight could run pbps with voting public players. There a lot of ways to add buzz to the playtest round without spending much in the way of resources.
*GASP* A Judge/Former SuperStar winners led Playtest!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
My tuppence / two cents / whatever is a penny or two in your area of the world... ;)
This is just to raise the thought for consideration, it may be groundless but I thought it better to be safe than sorry.
Culling earlier may result in culling something that hasn't been seen as often and may endanger a good item coming through the ranks as the vote spread evens out...
So, if you are going to cull earlier, maybe it could be based on culling items seen for an average "seen" count or greater and then only the bottom 5-10%, gradually culling say 5-10 times over a couple weeks (5 times at 10% culls, 10 times at 5% culls). I'm only raising this worry as I would hate to lose a good entrant/entry purely through a too early cull.
Cue the Python music...
And now for something completely different...
How about picking someone from the Blazing 9 each year for a particular nice piece of solid work that gets displayed there to do something small - this would give a number of advantages: notably
a) I think more people would practice and hone their skills here - that has to be number 1,
b) I found myself to be more objective and able to hone my own skills at assessing, critiquing and learning from everyone else, I'm sure others would similarly benefit
c) I counted down a lot to the next season via this thread, heightening my anticipation of the competition and getting my engagement earlier.
d) Those who do regular, consistent and solid use-able work might get spotted
I'm sure there may be other benefits, but those are the ones that immediately spring to my mind.
And finally, either...
Are you sure button - you hit submit, it returns like the preview button with an "are you sure you want to submit this" - in red if it has no title/excessive word count etc.
Or, Move the Submit Button to the Preview Screen so you have to hit Preview, and see what your entry looks like/with word count, at minimum once before you can Submit it.
I wouldn't have to Template Fu so much then :P
|
RPG Superstar 2013
,
Dedicated Voter Season 6, Dedicated Voter Season 7, Dedicated Voter Season 8, Star Voter Season 9
aka Steven T. Helt
|
| 1 person marked this as a favorite. |
*GASP* A Judge/Former SuperStar winners led Playtest!!!!!!!!!!!!!!!!
I dunno about that. I'd just use it as an excuse for a TPK. I was thinking they could make an intern do it.
Alter Round 1 to allow any magic item. Pure Wondrous Items have been done to death, but there are thousands of magic items out there that get banned because they're weapons or rods or armour. Frex, ISTR a mug that was kicked out because (amongst other things) you could hit someone with it, so it was a mace in disguise. Silly.
Post the rules for the later rounds, twists and all, much sooner. As it stands, people don't have much time to work out something good in the short interval between the reveal and the deadline. And if Real Life intervenes (as for Thorn Manor this year) someone suffers unfairly. I didn't bother to enter this year because I knew I wouldn't have time to make a decent showing in later rounds, simply because I have a full time job and family. But that sort of time pressure wouldn't disqualify one from producing an excellent winner's or finalist's module.
I suspect you're thinking of Cayden's Cup, which actually was in the top 32 in 2012. Clark was somewhat concerned about the mace-iness of it but it didn't seem to bother the other judges.
The comment field from voters would be the best addition to the contest. It would solve the problem of the critique thread where the items posted on the first page get more than 20 comments and the items posted after the third page get maybe 5 if they are lucky.
I think the optimal arrangement would be if after R1 the items and comments could be shown in a critique sub-forum if the contestant opts in, similar to how there used to be a special judges forum. That would take a significant amount of work to set up though, with the opt-in button and saving the comments. It would also solve the problem of the item critique thread being a colossal clusterf@%& like it was this year.
Yes to culling, as I proposed earlier, a first cull after 1 week would be nice.
More allowed inteaction from the contestants, I know previous years had some flame outs and DQs, but I had a better sense of the contestants and I think that translates to more interest.
Don't make the submission process any easier! Superstars don't DQ themselves and it makes the culling so much easier.
No more archetype rounds, because I suck at them ;-)
Barely. so no more archetype rounds!
And I want a ban on oozes :-)
If that's the case, Darkjoy, I vote for no more monster rounds! IIRC, there were no oozes allowed last year.
I'd be down to see a separate season for All-Stars pushing the idea of a year round competition season while partnering up with Adventures a Week and Kobold Press and any other 3PP that wants to run a competition.
After a few seasons you could have cross promotional challenges. Top 8 contesetents form superstar take on winners from Valhalla calling and pathmaster ect.
If that's the case, Darkjoy, I vote for no more monster rounds! IIRC, there were no oozes allowed last year.
Nah, just oozes. But seriously, monster and villain rounds show if the stat fu of the contestants is up to snuff. A vital skill for every freelancer.
I've been thinking about this, and I believe that if the intial voting round is shortened, only 1 cull should occur about half-way through. That way there is no real chance of an item that might be considered getting flushed. Two culls with a shorter period narrows the group too quickly.
I'd be down for Meepo's idea, too.
I would quite like to see this too, maybe even throwing all the judges into the fray too - an all voter made results BUT I wouldnt want to see it replace the annual competition that finds new talent every year.
| 1 person marked this as a favorite. |
I'd like to hear your thoughts on how we can improve the competition and the process for the next RPG Superstar competition. Ideas that are already floating about are:
* Shorten the voting period for R1. ....
* Implement the R1 culling earlier to avoid so much exposure to the really poor items. And perhaps add a second culling.
* Add a way to flag R1 ...
* Add a way for voters to add a short comment to an R1 item that remains hidden until the end of R1, at which time the author can see all the comments.
* Drop the playtest step from R4. ...No guarantees on any of the above, like I said, they're just ideas, and I'd like to hear about any other ways you think we can improve the competition.
For starters how about asking fans for input?
Not sure I can say how much I appreciate your asking. Paizo does this in spades, but thanks none the less. :)
-Yes to shortening the voting period*
-Yes to culling earlier OR culling twice* The have-you-seen-my-item-post-cull? thread seemed to generate interest. And it feels like it encouraged people to participate.
-Yes to flagging items. A drop down menu of 'why' is probably needed for the sake of those who have to screen them. An item might need to be pulled out of the voting or it will continue to be flagged. Or maybe a way to add an editor's note: .
-caveat Yes to comments. A person should have to opt in before crowd-sourcing comments start. A drop-down or bullet menu with SiaC, SAK, Grammar, not-publishable, etc.. This is probably coding prohibitive, but will remove a lot of the harsh comments most entrants are not ready for (whether they think so or not).
-Generally yes to cutting the playtest time, but I will post more later.
*all of these are mathmagically derived right? The data you must have been able to collect is incredible.
On the playtest cut. If folks post no feedback, then yes time should be reduced. If there was some kind of reward for posting playtest results the time could be more valuable. I like the suggestions SuperStar Helt made above and will expand add a few more.
*Skype famous GM-hosted games. James Jacobs & Erik Mona (etc.) tables always rank high on the PaizoCon lottery are a good examples as to why this would work.
*posting feedback is entering a drawing for (auto-graphed?) copy of the winning submission (or other Paizo swag).
*judges feedback on your item would be considered Paizo swag for most of us :)
*you-tube judges/Paizo playtests (sans comments).
*online play area hosted at Paizo.
*chronicle sheets for local PFS groups that participate/post (similar to novel chronicles or con-boons) (you could even ask previous Top 8 for help creating these :).
*standardized feedback form (survey monkey?)
*RPGSS Party. I am not sure how they do this but many shows offer a prize to different people/groups around the country who host their season premier/finale. I was planning to set up how-to-host-RPGSS-party after reading about Steve Miller's get together. I hosted one and it was NOT the way to host a party (I was so sure I was going all the way the year after being an alternate and hosted a party for my friends to critique my R2. :( It could even be something as simple as posting pics like ITLPD.
If somehow Paizo's VTT is done in time...that could be tapped for some Judge/Former Superstar/Paizo employee hosted playtests.
And could help generate interest.
| 10 people marked this as a favorite. |
On Charles' note, I would love it if Lisa (or some other woman in gaming) could be persuaded to be on the R1 judging panel next year. It would be fantastic if the gatekeepers weren't all guys for once.
- round 1, option to flag an item for breaking the rules should be reintroduced. Suggest that this opens up a form that allows the reviewer to tick which rule was breached and a box to include a short explanation, say 30 words, to justify the reason for flagging it.
- round 1, something in a contestants paizo profile confirming that their r1 submission is active in the contest, whether it has been rejected for a rule breach or whether it has been culled.
- I liked the way the paired items in r1 voting occasionally came up with comparable items eg. Gloves vs gauntlets, hat vs headband. It was useful to be able to compare like with like especially in later stages after the cull when you were more likely to see an item again.
I hesitate with this suggestion, and maybe its already done and I'm to anachronistic to notice but maybe utilize social media to promote more. Have a superstar twitter feed during the contest and later in developments after the superstar is "born.
Maybe have the top 4 write blog posts about their journey to the top of the round. I joked about reality TV but the things that make that successful can help you. Make us care about the contestants more and we'll be more in tune with the contest.
- Handle campaign setting requirements more elegantly
This is with regards to the River Kingdoms archetype round. I absolutely loved this twist, but I felt it could have been issued more elegantly. If you did not own the River Kingdoms guide, that placed you at a huge disadvantage and at risk of disqualification. The Inner Sea World Guide (the book that's expected for anyone using the Inner Sea campaign to own) gave little detail about the River Kingdoms.
Giving Guide to River Kingdoms was an awesome gesture. However, the RPG Superstar schedule demanded competitors get this very late in the process. I did not receive my copy of Guide to River Kingdoms until literally the night before the archetype deadline.
When I looked at the archetypes in R2, it seemed like the overall quality of the candidates suffered. Many of the solid archetypes had a tacked-on RK tie-in while ones that tried to fully embrace the theme looked like early drafts.
Overall, it might be best if the "twist" is announced much earlier, especially when you're considering reducing the time for R1.
Again, I loved the twist and want to see more rounds like this in the future.
I was thinking about this while walking the dog: I wonder whether having Round 2 be more "fluff" than "crunch" (i.e. last year's organizations vs. this year's archetypes) would be helpful.
While I enjoyed both, I think it's far easier for me as a fan to go through all the entries (and at 32, there are a lot to go through) and decide which is best if I'm just worrying about the writing and creativity without also trying to factor in whether it's balanced.
Also toward what Charles was saying, IIRC, there was some discussion in the thread about female contestants/entrants that women tend to enjoy the flavor aspects of the game more (granted, that's grossly generalizing). Thus having something that puts the focus on that more than mechanics might help keep a larger group interested. Since I assume Round 3 will continue to be monsters/NPCs (as I think it should!), people who are more focused on mechanics aren't likely to stop paying attention if Round 2 focuses a little less on that.
RPG Superstar 2013 Top 4, RPG Superstar 2011 Top 16
,
Dedicated Voter Season 6, Star Voter Season 7
aka primemover003
|
I hesitate with this suggestion, and maybe its already done and I'm to anachronistic to notice but maybe utilize social media to promote more. Have a superstar twitter feed during the contest and later in developments after the superstar is "born.
Maybe have the top 4 write blog posts about their journey to the top of the round. I joked about reality TV but the things that make that successful can help you. Make us care about the contestants more and we'll be more in tune with the contest.
Quite a few of the Top 32 have active Twitters (myself, Landon Cole, Chris Shaeffer via the Rule Zero Podcast). I think the no discussion rule kept some quieter than others, but I frequently hawked the RPG Superstar contest and Paizo to both my Twitter followers and Facebook friends. I know Steven has a large following in the Tulsa gaming community.
There was definitely a distinct drop off in the later rounds as the post counts clearly show from this year compared to last. This contest really engages the community but I do agree that the prohibition on contestants making comments really hurts. DQs in previous years also can hurt the contest so we obviously need to find a happy medium.
I had mentioned in the guildhall that more Custom Titles could be a nice incentive to stay involved over the long stretch of the contest. Some other ideas utilize the DMtools chatboard for a Q&A with the contestants. Featuring the public profile of contestants on the blog during the contest could steer more traffic to the contest as well. Perhaps featuring the PFS scenarios written by the runners up at conventions in the off season in a "This Could Be YOU!" manner could draw peoples interest?
--Vrock Superstar
|
RPG Superstar 2013
,
Dedicated Voter Season 6, Dedicated Voter Season 7, Dedicated Voter Season 8, Star Voter Season 9
aka Steven T. Helt
|
I dunno about the gag rule thing. In my specific case, there were several times where I feel people got something wrong about my entry and I wished I could make a brief comment to clarify it. But then a really valuable principle was revealed in that frustration: I have to improve in writing a description so those misunderstandings don't happen. If I were able to say "no, the text clearly says the thing can attack at A or C", I might not learn the lesson.
I agree that the rules are likely too strict and the contest could stand a little showmanship from contestants, but how much is too much might be a tough line to draw. The contest is pretty good as is. A few dynamic ideas will make it more fun for everyone, but it's also the best thing in gaming.
Next to a brutal tpk.
RPG Superstar 2013 Top 16
,
Dedicated Voter Season 6, Dedicated Voter Season 7, Star Voter Season 8, Star Voter Season 9
aka DLandonCole
|
| 1 person marked this as a favorite. |
To pick up on what Scott said, I didn't tweet much because I didn't want to run foul of the self-promotion rule.
I tend to agree with Joana - not having a single woman on the judge-o-matic was a bit odd.
I wouldn't reduce the voting time for round one too much - two weeks might be good - just because it's easy to have a week where you don't have the time or inclination to vote. A running total of the number of votes cast would be good.
In terms of rounds, I might suggest moving the monster round forward; it's one that sorts the sheep from the goats. Perhaps there are now enough archetypes that there is little design space and that's a challenge that could be rested for a bit.
Here's a fun one - free soundtrack download for all entrants of the Paizo staff singing this years song posts :P - when you submit, you get a reward link to that soundtrack :)