My Estranged Item: Or How I Learned to Stop Worrying and Love the Cull

[Monica Marlowe]

Oceanshieldwolf wrote:

* Then again, as random as Round 1, that doesn't explain how there are some folks who make it to Round 1 (and/or beyond) multiple times.

Sure it does, those people are consistently good at producing what Round 1 asks them to produce. If it were a random contest, we would draw lots, not provide creative applications. RPGSS is about finding talent and the cream of the crop. Cream rises to the top and so do the handful of multi-placed contestants who are talented in game design.

[Woody Elliott]

mamaursula wrote:

Brother Fen wrote:

And in my estimation the pit crew guys are borderline cheating. I didn't even tell the players in my group what my item was because they use the forums.

Cheating how?

I think the feeling among some people is that large groups of people working together and voting as a team skews the voting results. Even a 5% difference in the amount of votes an item gets will change it's final standing in comparison with other items as much as 10% (more realistically around 6.2% with the system that they use which mitigates the impact of group voting to a point) when the voting is set up like this. When you have pit crews that are able to have multiple people reach champion with every one reaching at least marathon, it does skew the voting and the purity of the competition to a point. Combine that with the ability and willingness to publically critique items in a forum through exceptionally thinly veiled code (everyone knows exactly what item people are talking about) in an attempt to sway public opinion and the competition might seem like less of a competition to some, and more of a good ole' boys club giving each other a quick hand up.

Having said that, remember how I said "some people" think that, I do not. Yes, I believe that pit crews skew the voting to an extent. The thing is, if your item is good enough to make the top 32, it will make the cut without a pit crew backing you. If it makes the cut, it is not the voting that decides who the top 32 are, it is the judges. While it may be easier for pit crew members to make the cut, it does nothing for actually making the important list. Since everyone is allowed to post their items for critique, it doesn't really matter if you made the cut or not if you did not make the round of 32. You still get feedback and hopefully learn from the experience and do better next year. It really changes nothing in the end.

The only thing that I have disliked about the process are the thinly disguised thrashing and/or promoting of specific items in the forums that are a blatant attempt to skew peoples opinions. I feel like "Popcorn" but it is an item the "Smells bad" and is "offensively named". I could go on but you get the idea.

[Monica Marlowe]

Woody Elliott wrote:

mamaursula wrote:

Brother Fen wrote:

And in my estimation the pit crew guys are borderline cheating. I didn't even tell the players in my group what my item was because they use the forums.

Cheating how?

I think the feeling among some people is that large groups of people working together and voting as a team skews the voting results. Even a 5% difference in the amount of votes an item gets will change it's final standing in comparison with other items as much as 10% (more realistically around 6.2% with the system that they use which mitigates the impact of group voting to a point) when the voting is set up like this. When you have pit crews that are able to have multiple people reach champion with every one reaching at least marathon, it does skew the voting and the purity of the competition to a point. Combine that with the ability and willingness to publically critique items in a forum through exceptionally thinly veiled code (everyone knows exactly what item people are talking about) in an attempt to sway public opinion and the competition might seem like less of a competition to some, and more of a good ole' boys club giving each other a quick hand up.

Having said that, remember how I said "some people" think that, I do not. Yes, I believe that pit crews skew the voting to an extent. The thing is, if your item is good enough to make the top 32, it will make the cut without a pit crew backing you. If it makes the cut, it is not the voting that decides who the top 32 are, it is the judges. While it may be easier for pit crew mebers to make the cut, it does nothing for actually making the important list. Since everyone is allowed to post their items for critique, it doesn't really matter if you made the cut or not if you did not make the round of 32. You still get feedback and hopefully learn from the experience and do better next year. It really changes nothing in the end.

The only thing that I have disliked about the process are the thinly disguised thrashing...

Please define "large groups" and "working together," I believe your perceptions fly far from the mark of reality. I would remind those that there are fewer than 10 Champion voters.

[Monica Marlowe]

Woody Elliott wrote:

The only thing that I have disliked about the process are the thinly disguised thrashing...

I will agree there, there was ONE [redacted] item and ONE [redacted]* and yet several people made incredibly unkind comments about both, among others. While I know I flagged them, they remained in the forum. There is no denying that those designers know exactly who they are and probably would have preferred to not have their items tried in the court of public opinion, rather than anonymously voted on.

*edited to remove unique items because I thought better of it since voting is not complete.

[Monica Marlowe]

I am just going to ask the group at large, what exactly do you think a "pit crew" is and how do you think it functions in the case of RPGSS?

I did not coin this phrase, nor can I say that it is what I would have called the 5 people I asked to proofread and critique my item, but now it's the word and we'll use it.

[quibblemuch]

mamaursula wrote:

I am just going to ask the group at large, what exactly do you think a "pit crew" is and how do you think it functions in the case of RPGSS?

I did not coin this phrase, nor can I say that it is what I would have called the 5 people I asked to proofread and critique my item, but now it's the word and we'll use it.

My understanding is that it is a group of people (of varying size, but fewer than, say, a decemviri) who, pre-submission, proofread and critique someone's item. As in all creative endeavors, no one succeeds in a vacuum, and it can be helpful to have another person look over one's work, even if only for typos. (After 15+ years as a professional writer, I trust my own ability to see typos in my work much less than when I started).

My understanding is not that it is some sort of elaborate voting tontine, wherein the last surviving member of the blood-sworn cabal agrees to share the contract for writing a module that is the ultimate RPGSS prize.

I'm fairly sure that the latter would be an ineffectual use of one's time as a creative professional (or would-be professional). And without knowing even a ballpark estimate of the number of voters and the average ratio of votes cast per voter, it seems to me that hurling around accusations of "cheating" or "tweaking the results" to be somewhat irresponsible.

But then, I'm an incurable optimist. Tally-ho, forward the future, more good work than time to enjoy it, and all that.

[frank gori]

Brother Fen wrote:

And in my estimation the pit crew guys are borderline cheating. I didn't even tell the players in my group what my item was because they use the forums.

1) It is not against the rules.

2) The spirit of this community is generally a positive one. Typically after the top 32 are announced we give each other feedback and try to lift one another up. Calling people borderline cheaters for behavior that is not a rules violation is against that spirit.
3) Guess what? my pit crew helps give each other feedback for things we submit everywhere including Wayfinder, and other 3PPs. Writing circles have existed for a loooong time and they aren't going anywhere for a very simple reason. You don't always smell your own farts. Design by committee is cheating, getting comments on your writing is simply smart.

Woody Elliott wrote:

The only thing that I have disliked about the process are the thinly disguised thrashing and/or promoting of specific items in the forums that are a blatant attempt to skew peoples opinions. I feel like "Popcorn" but it is an item the "Smells bad" and is "offensively named". I could go on but you get the idea.

I agree to a certain extent. This year the obfuscation was a little too weak. I for example could clearly identify 3 references to my own item and regularly guess other items being commented upon. Didn't really change my opinion.

I know I didn't comment in a way to defend anything from my pit crew or myself but I could see why it would be a concern. But the other alternative is radio silence during voting which would make the competition less fun.

On the joke item, they weren't going to make the cull. They never do well in voting for a very simple reason. Let's say this is a career you want and it is a dream of yours to work for your favorite gaming company and you have a job interview and in the waiting room are all the folk you're up against. One of them is dressed like Batman and another is dressed like a clown while yet another is openly smoking drugs in the waiting room. The first question in the interview is what you think of the other folk interviewing... Tell me right now you're not throwing those other candidates under the bus.

[quibblemuch]

GM_Solspiral wrote:

Tell me right now you're not throwing those other candidates under the bus.

I would never throw Batman under the bus, good sir. Never!

[frank gori]

For me a pit crew is a group that looks over each other's work to proofread and comment. My group has a private forum and shared google plus drive we use year round.

We don't hide the fact that we evaluate each other's work. We are very careful and respectful to the rules of the contest. We're also far from the only pit crew at work. I think the pitcrew's might even offset one another in effect...

[Clay Clouser]

I think I need a better pit crew...one doesn't vote at all and the other two just made star...

[Coleman]

As someone who doesn't use a pit crew, I can understand how someone might consider it borderline cheating. I think the original comment had that word "borderline" in there for a reason and I think that is an important distinction.

I myself don't consider anyone using a pit crew to be cheating...but I do believe that they have an advantage. While a person who is part of a crew may not always vote in favor of their crewmate's item, I can't see how at least some level of bias wouldn't take place. I think that's unavoidable, and such a bias may not even happen on a conscious level.

Personally I have two friends who I encouraged to enter, but they did not. If they had, I likely would have voted for their items over 99% of others. I think that's just human nature. And if I am lucky enough to advance, I am sure I'll ask for my Pathfinder playing friends to support my entry. Why wouldn't I? They're my friends, of course I'll ask for their support.

For those who were not lucky enough to have others to rely on for proof reading and editing, it's definitely a disadvantage. I think however that you have to try and do something about that. Rather than view it as others having an advantage, try and make it so that you are not disadvantaged. Form a group for next year. Get criticism and feedback from someone, even if they are not entrants themselves.

Everyone who is entering should seek every advantage that they can get.

[Kiel Howell]

A pit crew, writing circle, reviewers, whatever you want to call it, is a professional step that is necessary for any aspiring writer. There are very few who are good enough that an objective third party isn't necessary.

[Thomas LeBlanc]

I didn't have a pit crew. I showed my item to one person, who said they disliked part of the item. I kept that part anyway and submitted it. But I wish I had my wife proof-read it before I submitted.

I know exactly 1 person's item. And the only reason I know that, is that person told me exactly which item theirs was after they saw it on my keep list. I have yet to see my item on anyone's keep list.

Two people know my item, a marathon voter and a no star voter. I have seen my own item 9 times, 3 times pre-cull and 6 times after. Based what it has been paired against, not feeling my chances are too great.

"Champion Conspiracy"
If the voting numbers average out, that is .00225% of votes cast for my item. Not enough to make any real statistical advantage. If 10 champion voter work together and each each see the item 10 ten times and the total votes are 400k (see NOTE below), then that is only .025% of votes cast. (Assuming there will be 10 Champion voters, they vote together, and there are only 400k votes cast.)

NOTE:Average based on "almost a half million total votes cast" in for first year of voting and "more than 310,000 votes" last year. This year the voting period matches the first year in length, but no reduced voting timer later in the voting period.

[frank gori]

@Clay - freelance forge is a good place to shop, as are these forums, as is the blazing 9 thread.

@Coleman - good point about the word borderline, the value driven word there though is cheating. It's an insult to people's integrity. Even with the word borderline I'd be hard pressed to keep my temper in check if someone said that to my face or about my friends.

It is a very good idea to have someone else look over your work and if you think of it next year I'm more than happy to give you or anyone that asks for it some notes.

[Clay Clouser]

Haha. I'm good. My pit crew includes some great designers they just don't vote much.

[Coleman]

quibblemuch wrote:

Coleman wrote:

As someone who doesn't use a pit crew, I can understand how someone might consider it borderline cheating. I think the original comment had that word "borderline" in there for a reason and I think that is an important distinction.

I'm going to quibble with you on that one. It is a distinction, yes, but only to cover the speaker in case of being confronted. It's like saying a political opponent is guilty of "borderline graft" or that a professional athlete is guilty of "borderline doping" or a marathon runner is guilty of "borderline getting on a bus from mile 8 to mile 21."

When someone adds "borderline" to an accusatory word, when it is pointed out that the other person did no such thing, the speaker can always say "Hey, I just said borderline" - meanwhile, the insinuation has been effectively made and questions have been raised about the other person's integrity.

It's a pretty big bomb to drop, even mitigated by caveats and hemhaws, is what I'm saying.

Yeah, I agree with you to an extent. Although I cannot say what Brother Fen's actual intent was with his comment on pit crews, I did not read it as an accusation so much as a complaint. The validity of the complaint is debatable.

[Oceanshieldwolf]

mamaursula wrote:

Oceanshieldwolf wrote:

* Then again, as random as Round 1, that doesn't explain how there are some folks who make it to Round 1 (and/or beyond) multiple times.

Sure it does, those people are consistently good at producing what Round 1 asks them to produce. If it were a random contest, we would draw lots, not provide creative applications. RPGSS is about finding talent and the cream of the crop. Cream rises to the top and so do the handful of multi-placed contestants who are talented in game design.

Yes. That was the point. It's kinda random, but kinda also rewards those designers who can design within the popular mindset of the voters and appeal to the judges design parameter biases.

[frank gori]

The first person to coin the term "pit-crew" looks to be Anthony Adam. He has a "pit-crew" and is generally the most well liked poster in the community and he got cut in the cull.

[Eric Morton]

Rich Malena wrote:

I think what happens in the CMI thread should be celebrated, and I would be sad if that experience were somehow diminished in the name of refocusing on the competition.

I agree that the CMI thread should be celebrated. That's why I think it should happen before the Top 32 are revealed. Open up the CMI thread up to people who aren't being considered by the judges a week before the Top 32 are revealed so the CMI thread can be the main event for that week, then the Top 32 reveal can be the main event the following week.

In the past two years, here's what has happened: Public voting starts. At some point during voting, a cull occurs. Crowdsourcing informs several hundred people that they are probably not in the running. Voting then ends. For one week, nothing happens outside of the judges' chamber and no one is allowed to discuss specific items, even those we know with 99.99% certainty are not being reviewed by the judges. Then, on a Tuesday, the Top 32 are revealed. People spend twenty-four hours naming all of the items they would rather have seen in the Top 32. Then the Critique My Item thread opens and draws significantly more traffic than all of the Top 32 threads combined. A few days later, the judges compile and post a list of the Top 100 so voters who are wondering why their favorites didn't make the Top 32 can see that many of their favorites came close.

I propose that the process go more like this: Just like before, public voting starts, followed by a cull and some crowdsoourcing. Voting ends. Immediately after public voting ends, the judges post a list of the Top 100 items that will be considered by the judges. Everyone whose item is not on that official list is allowed to publicly post their item and ask for critiques. As the judges make their selections over the next seven days, the public participates in public critques and conversations about items everyone wanted to see in the Top 100/wants to see in the Top 32. A week later, once all of those activities have started to slow down a bit, the Top 32 are revealed.

Notice how the proposed order of events in the previous paragraph gives the voting public something meaningful to do every week of the contest. Instead of having a dead week after voting ends followed by two highly anticipated events occurring back-to-back, the proposed order of events has voting followed immediately by a Top 100 reveal and public critiques, followed a week later by the Top 32 reveal. That gives every step of the Superstar process time in the spotlight without having to extend the duration of the contest in any way. Just turn the dead week following the end of voting into a week where everyone not on the Top 100 list can ask for public critiques.

[Coleman]

GM_Solspiral wrote:

@Clay - freelance forge is a good place to shop, as are these forums, as is the blazing 9 thread.

@Coleman - good point about the word borderline, the value driven word there though is cheating. It's an insult to people's integrity. Even with the word borderline I'd be hard pressed to keep my temper in check if someone said that to my face or about my friends.

It is a very good idea to have someone else look over your work and if you think of it next year I'm more than happy to give you or anyone that asks for it some notes.

Thanks, SolSpiral, I appreciate it. You gave me some nice feedback on my item last year (the Boulder Thrower's Kit). I didn't rely on any proofreaders this year, but normally I would. My gaming group world normally fill that role. Time constraints this year prevented that.

However, the community here on the boards is nothing short of impressive. I have always been more of a lurker in the past, but resolved this year to try and be a bit more involved. I would feel comfortable asking a number of posters here for advice. I likely will do so in the future, so don't be surprised if I take you up on that.

Thanks again!

[frank gori]

Coleman wrote:

Thanks, SolSpiral, I appreciate it. You gave me some nice feedback on my item last year (the Boulder Thrower's Kit). I didn't rely on any proofreaders this year, but normally I would. My gaming group world normally fill that role. Time constraints this year prevented that.

However, the community here on the boards is nothing short of impressive. I have always been more of a lurker in the past, but resolved this year to try and be a bit more involved. I would feel comfortable asking a number of posters here for advice. I likely will do so in the future, so don't be surprised if I take you up on that.

Thanks again!

I remember that item fondly, I even used it for an NPC villain.

[Coleman]

GM_Solspiral wrote:

I remember that item fondly, I even used it for an NPC villain.

Ha very cool! Thanks man.

[Oceanshieldwolf]

Eric Morton wrote:

I propose that the process go more like this: Just like before, public voting starts, followed by a cull and some crowdsoourcing. Voting ends. Immediately after public voting ends, the judges post a list of the Top 100 items that will be considered by the judges. Everyone whose item is not on that official list is allowed to publicly post their item and ask for critiques. As the judges make their selections over the next seven days, the public participates in public critques and conversations about items everyone wanted to see in the Top 100/wants to see in the Top 32. A week later, once all of those activities have started to slow down a bit, the Top 32 are revealed.

Notice how the proposed order of events in the previous paragraph gives the voting public something meaningful to do every week of the contest. Instead of having a dead week after voting ends followed by two highly anticipated events occurring back-to-back, the proposed order of events has voting followed immediately by a Top 100 reveal and public critiques, followed a week later by the Top 32 reveal. That gives every step of the Superstar process time in the spotlight without having to extend the duration of the contest in any way. Just turn the dead week following the end of voting into a week where everyone not on the Top 100 list can ask for public critiques.

[Emphasis mine for awesomeness] Can't favorite your post enough Eric. Nice.

[frank gori]

re: Eric's idea: I agree this would be cool as long as the top 100 are the only items considered... I have no idea if the golden ticket still exists.

If not it does solve the problem of a dead week

[Adam Daigle]

As far as I know, the golden tickets still exist... which would make Eric's suggestion difficult.

[Adam Daigle]

Also, when Eric's idea was first presented, it was couched in the idea of something that would help focus on the Top 32, but as it's being discussed more and more, I'm seeing less of that focus. It looks more like a way to keep the voting public engaged and a way to help focus on everyone but the Top 32. What am I missing?

[Eric Morton]

Regarding pit crews:

Spoiler:

In my seven years competing in Superstar, I haven't once run anything I've submitted past anyone. Everything went straight from my brain to my computer to the Superstar entry form with no one else reviewing it. I just assumed that's how everyone else was approaching the competition. This year, though, I've see a whole bunch of comments about "my pit crew said" this, and "I saw an item from my pit crew" that. I was entirely surprised by the prevalence of pit crews.

That being said, there are no rules against groups of friends having private conversations about specific items. Utilizing a pit crew isn't borderline cheating (whatever "borderline cheating" is); it's straight-up, 100% allowed by the rules.

[Coleman]

My concern with that would be how many folks would stop participating once they got some feedback on their item. I know that one year I totally blew off the rest of the competition once the top 32 were announced and I was able to get feedback on my item that didn't make it. I didn't participate in any of the subsequent rounds. I imagine it is the same for others.

Since then I resolved to see things through to the final round. I would imagine that if people are still waiting for or getting feedback when the top 32 are announced, they'll at least vote.

Other than that though, I think Eric's idea has a lot of merit.

[quibblemuch]

Eric Morton wrote:

(whatever "borderline cheating" is)

Maybe it's some kind of ethical 5-foot step that you can make without provoking attacks of opportunity on your integrity? Or maybe it's the moral compliment to "borderline following the rules"?

Oh! I got it! It's when you "accidentally" spill your Diet Coke onto the battle mat and then scuff out the lines and point to the wrong place on the map so the flustered GM redraws the room to a more tactically advantageous configuration!

Of course!

Well heck, we've all done that, right? Right?

[Eric Morton]

Adam Daigle wrote:

Also, when Eric's idea was first presented, it was couched in the idea of something that would help focus on the Top 32, but as it's being discussed more and more, I'm seeing less of that focus. It looks more like a way to keep the voting public engaged and a way to help focus on everyone but the Top 32. What am I missing?

Many contestants want to participate in the critique thread and follow the Top 32 threads. By revealing the Top 32 and opening the critique thread in the same week, you are forcing lots of voters to divide their limited forum viewing time between those two different parts of the Superstar contest. "Do I spend my limited forum time soliciting and reading feedback on my own item, or do I pass up the opportunity to get valuable feedback and instead pay close attention to the Top 32 threads?"

Think about it this way: How much attention would the Top 32 get if the Top 32 reveal was on Tuesday and a Pathfinder Unchained open playtest started on Wednesday? Sure, lots of people who are going to participate in the playtest also want to pay attention to the Top 32, but will they have the time? The annual Critique My Item thread is like that hypothetical open playest in that it attracts lots of attention.

I suggest that an event that attracts lots of attention should not be scheduled to run concurrently with the Top 32 reveal because that lessens the impact of the Top 32 reveal, and that moving the Critique My Item thread up a week will annoy fewer people than delaying it a week. Also, I doubt that moving the Critique My Item thread up a week will do anything to increase the anticipation contestants have for that thread; based on posts I'm seeing in this forum, anticipation for that thread is already running rather high.

[Eric Morton]

Coleman wrote:

My concern with that would be how many folks would stop participating once they got some feedback on their item.

I would not take people who want to leave the contest after getting feedback on their own items into consideration when scheduling things. The contestants who are only in it to get feedback on their own items aren't going to care very much about the Top 32 reveal, no matter when you schedule it. I'm more concerned about the subset of contestants who want to get/give feedback on non-finalist items and want to follow the progress of the Top 32. The current schedule forces these contestants who are interested in every facet of RPG Superstar to divide their attention between the non-finalist critique thread and the finalists' threads during the week of the Top 32 reveal, which can only serve to reduce traffic visiting the Top 32 threads.

[Coleman]

Eric Morton wrote:

Coleman wrote:

My concern with that would be how many folks would stop participating once they got some feedback on their item.

I would not take people who want to leave the contest after getting feedback on their own items into consideration when scheduling things. The contestants who are only in it to get feedback on their own items aren't going to care very much about the Top 32 reveal, no matter when you schedule it. I'm more concerned about the subset of contestants who want to get/give feedback on non-finalist items and want to follow the progress of the Top 32. The current schedule forces these contestants who are interested in every facet of RPG Superstar to divide their attention between the non-finalist critique thread and the finalists' threads during the week of the Top 32 reveal, which can only serve to reduce traffic visiting the Top 32 threads.

I think you make a valid point in that regard. I would say that this idea would be worth considering on Paizo's part.

[Felinus]

The alternative would be to close CMI till the close of the competition. Who wants to wait that long?

[Cthulhudrew]

Eric Morton wrote:

The way the contest is set up right now, for hundreds of people, the Top 32 reveal is just a speed-bump on the way to the Critique My Item thread. Everywhere I look in this forum right now, I am seeing posts that essentially say, "Looks like my item got culled. I can't wait to post it in the Critique My Item thread."

But that is not anything to do with the way the contest is set up, nor the purpose of it, and everything to do with participants own (misguided?) expectations about what the contest is for.

If all people are worried about is, "please critique my item," there are numerous places elsewhere on the forums for them to engage in such creative, communal feedback that aren't connected to the contest, and won't diminish the point of the whole competition.

It's simply a matter of putting the hardwired "risk/reward" mentality on the backburner* and refocusing on the clear, stated purpose of RPG Superstar. It's not for immediate gratification, and if that's what people need and/or crave, then I think they're looking in the wrong place.

(*Or, as noted above, finding an alternative creative outlet to engage in design and critique, and just waiting to share your item in the meanwhile.)

[Nickolas Floyd]

I'm not here calling anyone cheater. Far from it. I don't feel that having a pit crew who gives you feedback on your item prior to submitting is cheating at all. I showed my friends my items and asked them what they thought. Of course only one person in my pit crew actually enters and votes in this competition, and I only know him through the forums. I just need more and better friends, I guess.

But I don't think it is cheating either to consciously or unconsciously favor your pit crew's items in the voting either. That is helping your social group, and is just human. My suggestion that it might be possible for these groups to effect the top 100 the judges will view by voting together and voting a lot, was a criticism of the larger cull. A smaller pool of items to vote on makes each vote more powerful. My suggestion that they could negative vote was just throwing that out there. I doubt that there is any group out there who would do this on a mass scale, but human voters might down vote items that might seem similar to their own (or a buddy's) item.

A smaller cull does not "speed things up" at all as some have suggested. It is still the same amount of voting time and the judges will only look at the top 100 or so items. It does make the item pairing better to be sure, but just about everyone has seen an item that they were surprised made the cull, and items that were on their favorite list that did not make the cut. I think a large cull likely removed voters from the voting group or at least slowed down their voting enthusiasm, again giving more power to the pit crew groups, even if some of the group got culled. I don't know if this is in fact the case, but I think it is possible and should be something that the contest staff should be aware of. Not knowing the voting statistics, as we don't, it would be impossible to know either way, and depending on what all is recorded behind the scenes, it might not be able to be seen there either, at least until anonymity is dropped when the top 32 are announced. If it is a problem, it is not the pit crews' fault. They are not "cheating." They are just taking advantage of the system that is in place, possibly without intent. That is of course unless your pit crew is systematically negative voting. That's just evil.

[varathiel]

I think pit crews are unfair under the current rules. You are right that the rules don't prohibit it. It is even fairly appropriate considering that Pathfinder, and 3.X in general, is like catnip for rules lawyers.

However,

The rules don't explicitly allow it either. If it is the intention to allow it, then they should say as much. For first timers, such as myself, it is really easy to read those rules and assume that you can't get help from people on the forums. This creates an unfair advantage for previous contestants. I'm OK with pit crews, but it should say something like this in the rules:

"It is allowed, and strongly recommended, that you show your item to others for critique before entering it, as long as you do not do so publicly."

This would let first timers know that this practice is acceptable. Those who choose not to are taking the disadvantage willingly at that point.

I can't speak for the other first timers, but I was shocked when I saw people talking about their pit crews because I assumed that we couldn't talk about our items at all. Proofread my item myself and then submitted it. If I had known that it was allowed, I would have availed myself of the opportunity.

I am still in and still hopeful that I make the top 32, but I have seen my item mentioned a few times along the lines of, "why did you use x instead of y?". I have a great answer for that that I am not allowed to say for fear that I will be disqualified if I am not vague enough. This has caused another worry for me, so let me ask this question to those of you with pit crews (and this is not accusatory in anyway, simply my own curiosity): Do you share your keep lists with your pit crew and/or discuss items behind the scenes?

I don't think it's wrong if you do, but if you have a negative or positive opinion of an item does it not become possible to sway the opinions of your pit crew on items whose creators are not allowed to defend their decisions?

[Cthulhudrew]

Adam Daigle wrote:

Also, when Eric's idea was first presented, it was couched in the idea of something that would help focus on the Top 32, but as it's being discussed more and more, I'm seeing less of that focus. It looks more like a way to keep the voting public engaged and a way to help focus on everyone but the Top 32. What am I missing?

That's what I'm hearing, too. And it just seems completely against the purpose of the contest. (For clarification: The purpose of the people who put on the contest, not the purpose that the public may have come to view it as.)

[Rich Malena]

Adam Daigle wrote:

Also, when Eric's idea was first presented, it was couched in the idea of something that would help focus on the Top 32, but as it's being discussed more and more, I'm seeing less of that focus. It looks more like a way to keep the voting public engaged and a way to help focus on everyone but the Top 32. What am I missing?

Silly suggestion, but once open voting stops, is there any reason why we can't lift the anonymity gag order? The community can start critiquing everything, and then the judges arrive from their secret conversations and we all find out which items get elevated? Keeps the excitement going all the way up to the big reveal, and the reveal is still a huge important moment? We talk about our favorite items before the big day! It's like me making fun of Ohio States fans all the way to Monday night!

I mean, the judges would have to be sequestered for a bit, but we're raring for the next step!

[James Casey]

Cthulhudrew wrote:

Eric Morton wrote:

The way the contest is set up right now, for hundreds of people, the Top 32 reveal is just a speed-bump on the way to the Critique My Item thread. Everywhere I look in this forum right now, I am seeing posts that essentially say, "Looks like my item got culled. I can't wait to post it in the Critique My Item thread."

But that is not anything to do with the way the contest is set up, nor the purpose of it, and everything to do with participants own (misguided?) expectations about what the contest is for.

If all people are worried about is, "please critique my item," there are numerous places elsewhere on the forums for them to engage in such creative, communal feedback that aren't connected to the contest, and won't diminish the point of the whole competition.

It's simply a matter of putting the hardwired "risk/reward" mentality on the backburner* and refocusing on the clear, stated purpose of RPG Superstar. It's not for immediate gratification, and if that's what people need and/or crave, then I think they're looking in the wrong place.

(*Or, as noted above, finding an alternative creative outlet to engage in design and critique, and just waiting to share your item in the meanwhile.)

If the CMI thread is more popular and receives more attention than the Top 32 and all other subsequent RPGSS threads, that should tell you more about the voters and what's important to them. Trying to "prop" up RPGSS by delaying the CMI thread isn't the solution. Those people aren't truly interested in following RPGSS unless they are participating.

I argue this defines the majority of the voters and back it up by the number of posts in the RPGSS threads from past years vs. the CMI posts. I also argue that the main benefit of RPGSS to both the community and Paizo diminishes significantly after each round. The largest benefit of the competition is to motivate hundreds, maybe thousands, of people to think creatively and possibly think about designing for this wonderful game. The benefit is to introduce new designers to all of the companies that produce material for this game. It's only natural that as more people are eliminated, their interest wanes in the contest and turns toward improving themselves, which the CMI thread facilitates. How many published designers have been found through this competition, yet never made Top 32? How many published designers that don't follow the contest once they were eliminated have been found through this competition? I don't know the answers but I bet they would surprise most people.

The people who are interested in following the competition won't have their interest distracted because of the CMI thread. However, they may be distracted because they are interested in following both threads. Following Eric's suggestions, or some hybrid, is a great idea that I feel should be explored.

[Cthulhudrew]

James Casey wrote:

If the CMI thread is more popular and receives more attention than the Top 32 and all other subsequent RPGSS threads, that should tell you more about the voters and what's important to them. Trying to "prop" up RPGSS by delaying the CMI thread isn't the solution. Those people aren't truly interested in following RPGSS unless they are participating.

I certainly am in agreement about the people who weren't really interested in RPGSS.

However, I'm not sure the argument about the CMI thread being more popular and receiving more attention has merit. I haven't run any numbers on combined posts, but my questions about that argument would be thus:

1) Are there really more posts in that thread than in the combined top 32?
2) Are there more substantive feedback posts than are in the top 32 threads? (ie, no "Me too's!")
3) How does the number of active participants in that thread compare to, say, the pre-32 threads (such as the "So Far I've Seen"). Is it a substantial number, a small portion of mostly the same faces?
4) Who are the people most actively giving feedback there*?

And, most importantly,

5) If there is greater interest in that thread, does the greater participation there somehow equate to a lack of voting in the remainder of the competition? Just because there isn't that much commentary going on, does not necessarily equate to drop in votership. We never see those numbers, only Paizo does/could.

If the Critique My Item thread has somehow become the most important part of this competition to many (most?) then perhaps that is the real issue, and some alternative can be found for those people to actively and consistently participate in that sort of thing, and divorce it from the competition altogether?

(*For instance, having a forum where the people largely giving feedback from the CMI thread have their own, ongoing, presence to potential designers.)

[Oceanshieldwolf]

I agree completely James. Events are not set in stone, nor defined only by their existence at one time. Like communities and social groups, their needs and processes change over time. I would argue that RPGSS has changed, and that the change deserves to be recognised.

What that change is becomes the purview of Paizo. We can only offer opinions and make suggestions.

[Oceanshieldwolf]

Cthulhudrew wrote:

If the Critique My Item thread has somehow become the most important part of this competition to many (most?) then perhaps that is the real issue, and some alternative can be found for those people to actively and consistently participate in that sort of thing, and divorce it from the competition altogether?

(*For instance, having a forum where the people largely giving feedback from the CMI thread have their own, ongoing, presence to potential designers.)

I agree with this also.

[Cthulhudrew]

Maybe we should have a "Pit Crew" forum or thread or something as an ongoing (ie, not solely confined to RPGSS) place. I know there are already forums here where people go for that kind of feedback, so perhaps a sub-forum in the "Advice" forum or something?

[Oceanshieldwolf]

Just sounds like the Suggestions/Houserules/Homebrew sub-forum, just with greater in-thread moderation, possibly like the Multiclass Archetypes threads, pretty much moderated solely by Elghinn Lightbringer for @3.5 years over 9 threads and @10,000 posts.

A committee of interested critiquers could set it up, much like is already done with Blazing Nine threads....

[Isaac Volynskiy]

You can post stuff for feedback in Homebrew all year!

If you want to have an item reviewed for RPGSS, then make friends with the forumites so you can privately discuss it.

[Jacob W. Michaels]

Cthulhudrew wrote:

However, I'm not sure the argument about the CMI thread being more popular and receiving more attention has merit. I haven't run any numbers on combined posts, but my questions about that argument would be thus:

1) Are there really more posts in that thread than in the combined top 32?
2) Are there more substantive feedback posts than are in the top 32 threads? (ie, no "Me too's!")
3) How does the number of active participants in that thread compare to, say, the pre-32 threads (such as the "So Far I've Seen"). Is it a substantial number, a small portion of mostly the same faces?
4) Who are the people most actively giving feedback there*?

Last years' CMI thread had 733 responses, which doesn't count a few people who confined their responses to separate threads (GM_Solspiral's GBU reviews, Haladir...).

The Top 32 averaged 29 responses, ranging from 10 to 42. Totaled, that'd be about 928, if I did my math correctly.

I assume there are far more active participants because there are far more than 32 people who don't make it into the contest and thus want to get feedback.

I'll get more into this in my blog post Tuesday, but the number of comments in subsequent rounds have dropped precipitously -- by more than a half -- since public voting began.

[Eric Morton]

Cthulhudrew wrote:

However, I'm not sure the argument about the CMI thread being more popular and receiving more attention has merit.

If the CMI thread does not generate more attention than the Top 32 reveal, then moving it up a week will give voters something to do in the week leading up to the Top 32 reveal without reducing the anticipation for the Top 32 reveal.

If the CMI thread does generate more interest than the Top 32 reveal, then moving it up a week will allow the Top 32 reveal to be the central event of the week during which it happens without the CMI thread being a distraction.

Either way, I'm not seeing any drawback to a system that schedules exactly one highly anticipated Superstar milestones per week (voting starts, first cull, second cull, voting ends, critiquing starts, Top 32 reveal, etc.).

[James Casey]

The Blazing Nine Thread exists throughout the year which fills the niche you are discussing, but the catalyst for that thread and the associated creativity is the competition. Without the catalyst, threads like TB9 and CMI have no food to fuel them.

For many would be designers and even those interested in only learning about design for their home game, RPGSS serves as fuel for the fire, the catalyst. From the competition threads like TB9 and CMI are created where individuals can go to receive non-threatening (usually) critique from their peers. It also provides them with validation.

I don't think the only reason Paizo has this competition is to find one new designer. In fact I don't think it is the primary reason. As I mentioned earlier, it is to foment creativity in the community. It also serves to foster incredible amounts of good will between Paizo and the players and between players and other players, which is why people get angry about the snark thread, because it is perceived as flying in the face of the good will engendered by the contest. I argue that it is merely a forum for venting one's voter fatigue which is why I got upset and stopped posting this year because I felt it singled out too many entries in a specific manner, but that is an entirely different conversation/thread.

I guess what I am saying is the contest isn't only about the Top 32, far from it. It is obvious to me that you care a great deal about the contest and all of the subsequent rounds. So let me ask you a question.

Does the CMI thread deter you from voting in the competition?

If the answer is no, then there is no need to worry about it overshadowing the Top 32.

[James Casey]

Jacob W. Michaels wrote:

Cthulhudrew wrote:

However, I'm not sure the argument about the CMI thread being more popular and receiving more attention has merit. I haven't run any numbers on combined posts, but my questions about that argument would be thus:

1) Are there really more posts in that thread than in the combined top 32?
2) Are there more substantive feedback posts than are in the top 32 threads? (ie, no "Me too's!")
3) How does the number of active participants in that thread compare to, say, the pre-32 threads (such as the "So Far I've Seen"). Is it a substantial number, a small portion of mostly the same faces?
4) Who are the people most actively giving feedback there*?

Last years' CMI thread had 733 responses, which doesn't count a few people who confined their responses to separate threads (GM_Solspiral's GBU reviews, Haladir...).

The Top 32 averaged 29 responses, ranging from 10 to 42. Totaled, that'd be about 928, if I did my math correctly.

I assume there are far more active participants because there are far more than 32 people who don't make it into the contest and thus want to get feedback.

I'll get more into this in my blog post Tuesday, but the number of comments in subsequent rounds have dropped precipitously -- by more than a half -- since public voting began.

Let me amend my earlier statement- the number of posters rather than the total number of posts. And Jacob is correct. Each round saw less and less individual posters. Fatigue obviously sets in and interest wanes.