Want to be a better designer? Start by being good at workshopping

[frank gori]

I'm bloggy lately but no one talks about this so here you go This is about being a better pit crew ally

[Curaigh]

Thanks Frank, Nice record this year. :)

Limiting critique to two or three points is excellent advice. While you might have plenty more, it doesn't do much good to change every verb to active voice if the entire paragraph will be cut for reasons of SAK. Hopefully too, a writer has more than one crew member so it will all get caught eventually. For others who would like workshop practice stick around. We got you covered.

I have often wanted to ask Top 32 to 'critique the critique.' I wonder if I am doing any good to those I critique (and due to the Blazing 9 I critique quite a bit. :) I hope you don't mind if I piggy back on this thread so....

Hey previous Top 32, please pick a critique of any of your entries. Copy & paste it here and then tell us why it was helpful or not helpful. That's a lot to ask, I know, but we have a long stretch until the next votes are due. :)

[Shifty]

Spot on too.

Everything you say can be applied to the start-up space, and business in general.

[frank gori]

Curaigh wrote:

I hope you don't mind if I piggy back on this thread so....

Hey previous Top 32, please pick a critique of any of your entries. Copy & paste it here and then tell us why it was helpful or not helpful. That's a lot to ask, I know, but we have a long stretch until the next votes are due. :)

cool idea I'll participate in that for sure

[The Raven Black]

I am quite new at helping contestants through workshopping and I was met with a daunting challenge. I would like to know how other people tackle it.

One of the person I helped workshopping for the monster round is now in the Top8 and asked me for help reviewing an encounter, which I did.

I already had trouble with keeping my review of the monster unpolluted by my own design ideas. I was easily finding ways to improve the monster I was reviewing but I adamantly did not want to mix these ideas with those of the contestant.

I decided to solve this conundrum by making comments on the facts of the monster and trying as much as possible to ask questions about points I think needed to be addressed, rather than give the answers I would have given if I were in the contestant's place.

Because I just love using my imagination to fill the gaps seamlessly and it does come very easily to me. But in such an exercise, this becomes quite the hindrance.

And if that was already hard in the monster round, it has become terrifying when reviewing the encounter.

How do you, experienced workshoppers, do this ?

[Neil Spicer]

The Raven Black wrote:

I already had trouble with keeping my review of the monster unpolluted by my own design ideas. I was easily finding ways to improve the monster I was reviewing but I adamantly did not want to mix these ideas with those of the contestant.

This has been my concern with workshopping from the very beginning. It's one thing to have a "sounding board" to bounce ideas off of and get reactions to your work, but "workshopping" is more of a group-design concept. Workshopping (at least in the literary sense) generally involves sharing your work and then getting both feedback and concrete suggestions on how to improve it. When Sean, Clark, Ryan, and I did the RPG Superstar panel discussions at PaizoCon, there would inevitably be a question about getting design feedback from others before submitting to the contest. Clark would always caution against going too far with that. Otherwise, was the design output truly yours or was it a reflection of how well you could participate in a group design? While group design has its place (if you're working in-house for a publisher or on a group project), RPG Superstar is mostly seeking to identify heretofore unknown talent at an individual level. So, with that principle in mind, competitors and potential workshoppers need to make sure the end result is reflective of the individual's Superstar-caliber talent. If you want to learn and educate yourself further on different kinds of designs, the best place for that is workshopping ideas outside of the competition (e.g., in the Blazing 9 thread, etc.), and then, when the contest rolls around, have someone who can give you their reactions to your design moreso than directions on how to improve it. Otherwise, they're redesigning your item for you, and it's not necessarily indicative of what you can do on your own. And, if you're always relying on others to do that kind of stuff for you, you're not necessarily stretching yourself the way you need to in order to become a Superstar designer which a publisher can rely on for major project assignments.

But that's just my two cents,
--Neil

[frank gori]

Neil Spicer wrote:

This has been my concern with workshopping from the very beginning. It's one thing to have a "sounding board" to bounce ideas off of and get reactions to your work, but "workshopping" is more of a group-design concept. Workshopping (at least in the literary sense) generally involves sharing your work and then getting both feedback and concrete suggestions on how to improve it. When Sean, Clark, Ryan, and I did the RPG Superstar panel discussions at PaizoCon, there would inevitably be a question about getting design feedback from others before submitting to the contest. Clark would always caution against going too far with that. Otherwise, was the design output truly yours or was it a reflection of how well you could participate in a group design? While group design has its place (if you're working in-house for a publisher or a group project), RPG Superstar is mostly seeking to identify heretofore unknown talent at an individual level. So, with that principle in mind, competitors and potential workshoppers need to make sure the end result is reflective of the individual's Superstar-caliber talent. If you want to learn and educate yourself further on different kinds of designs, the best place for that is workshopping ideas outside of the competition (e.g., in the Blazing 9 thread, etc.), and then, when the contest rolls around, have someone who can give you their reactions to your design moreso than directions on how to improve it. Otherwise, they're redesigning your item for you, and it's not necessarily indicative of what you can do on your own.

But that's just my two cents,
--Neil

I completely understand the concern from a contest perspective. That said, having written things inside and outside the industry and having more then a few friends that also write, workshopping is something most authors do. It's hard to kill your babies but it becomes easier when someone points it out to you.

For my own part the rules I follow for the contest is that suggestions are welcome but no editing. Someone can suggest a direction (bounce) all day as long as at the end of that day it is still the author's work. Heck that's no different then talking about your idea with someone.

Outside the contest, my company designs as individuals and edits as a group which is extremely helpful as we learn from and play of each other. We also do it for our freelancers if they want eyes on their work outside our organization. In the end it improves the overall quality of the work handed over which tends to be appreciated.

[Taylor Hubler]

I think there should be a discussion on effective workshopping for the contest and what ineffective workshopping could be. So far the arguments have been about for or against workshopping, but there hasn't really been much of a detailed talk about what people are doing, how the actions taken, and how the different parts of workshopping effect the submissions.

Frank did a pretty good job about how to be part of a workshop but I would love to see more people talk about their processess and direct experiences. It is possible that there is a right and a wrong way to do this, and that those arguing on either side are coming from a narrow point of view and are making incorrect assumptions about process.

My personal process involved controlling the process as much as I could. I learned about workshopping as I went along in the contest and refined my process with each step, but I feel I have found a way to do workshopping that helps me as a writer and a contestant without compromising the legitmacy of my submission or risking my own voice being taken from the piece.

• Keep the number of people looking at your piece down to a minimum. I was going to be workshopping with only three or four people if I had moved on to the encounter round, each one picked because I knew I could trust them, and because I knew they had knowledge and insights I could benefit from.
  
• Share the piece privately and in a way so that only you can directly edit it. I used Google Docs and when I shared the piece with people and I made sure they only had permission to comment on it and not do anything else. This meant that if a change was needed, I did it myself and did it with my voice. It also meant that I was following the rules of the contest and that no one else had done work for me.
  
• Don't share it with everyone all at once. I wouldn't share it with the whole group of people at one time. Share it with one or two people first and then bring in the rest. This keeps the amount of commentary down to a manageable amount, and if someone has an area of specialization (such as templates or rules language,) you can better focus them onto those areas.
  
• Don't feel like you have to apply all of the comments, but keep in mind that your piece has to stand up on its own without your defence or clarifications. I certainly should have listened to people on a couple points when it came to my monster, but I also held my ground on a couple details that ended up working out for me. The balance is found in asking yourself about how well things stand up on their own. If everyone is pointing out a trouble area, you need to give it attention. If one person is expressing their personal tastes, you could probably leave it alone.
  
• Cut people off from the document once you have submitted your piece. This removes the conversations you had with them and prevents them from going back to find your own clarificaitons. It also keeps them from directly quoting you in public, which could get you DQed.
  
• Don't clarify your own piece to the people workshopping it unless you need advice on a process. I was guilty of clarifying, but in a couple instances I should not have said anything. If people didn't get it I needed to make revisions instead of clarifying. The submission has to stand up without your commentary, so practicing retraint in workshopping will help you later.
  

I think this process should be a good one for contest level workshopping. I am sure I missed some part of my own process but I did get the important parts.

[frank gori]

Taylor Hubler wrote:

What Taylor said

Could not like that hard enough +1

[Jacob W. Michaels]

I've been asked to help a lot of people in the last two seasons, which I enjoy doing, as it keeps me involved in the contest even when I'm not in the Top 32.

To answer The Raven Black, truthfully, it can be a challenge to tread the line. I enjoy doing design work and want to give the best advice I can to people who are trusting me. At the same time, I believe very strongly in this contest and that it's important the competitors are submitting their own designs and not mine. Really it's simply a matter of self-control and as Neil notes reacting to work and not doing the work.

--

I think Taylor brings up a lot of good points of what a competitor is/should be doing. I especially agree with keeping the number of people looking at your designs down; otherwise you just run into paralysis by analysis.

As someone giving feedback, I think one of the most important things is being prompt. You may give me the best feedback ever, but if you wait two days and give it to me 8 hours before the submission deadline, that's next to useless (and in some ways worse because then I start worrying about choices at a point I may not be able to fix them). I try to respond quickly at all hours of the day.

[Neil Spicer]

In the interests of sharing my process, during RPG Superstar 2009, I only had two people I consulted on my idea. Never my submission. In essence, I described what I was thinking about doing with one gamer friend who's never done design work professionally or even semi-pro as a freelancer. He was a player in my home campaign for nearly 6 years and I simply trusted his judgment on what might be a compelling, interesting idea to generate excitement in an everyday gamer (and voter). I never showed him my actual submission. We only ever talked in terms of an idea I had, and he only ever saw the end result when it became available for the voters.

The second person I occasionally consulted was my wife...a total non-gamer. But, I figured if I could discuss my idea with her and it piqued her interest, as well, I was potentially onto something that might attract new gamers in addition to veteran gamers (such as my friend, above). And that's it. No true workshopping. Just reactive feedback to the ideas I had under consideration and then a bit of discussion to help me discern the strongest of them before getting down to work.

From there, I always did the writeup and/or mechanical tinkering for my designs on my own. I tightened it up as best as I could, applying my limited-but-growing understanding of the rules, and then I summoned as much courage as I could and submitted it. From there, the true "workshopping" I received came from the comments people posted in the eventual submission thread, and that's why I always paid such close attention to it. I'd tweak my approach round-by-round to win them over as voters with the next design, shoring up weaknesses as they were identified, or playing to strengths based on what people seemed to really like.

So, call me biased or self-promoting or hardcore, but I think that's a far better method of building yourself into a Superstar designer than workshopping it with a group. To me, the latter is something you can do as an educational process outside the competition. But, to do it during the competition...i.e., to workshop by sharing your full submission with a group of fellow designers (both experienced in RPGSS or still trying to get their foot in the door)...isn't reflective of your own ability. It's a combination of yours and theirs. Sure, there are some projects and freelancing guilds who'll combine forces in order to improve themselves like that. And, you're always going to have a developer and editor cleaning up whatever work you eventually turnover in the industry. But, for RPG Superstar...to discern the best of the best...I like knowing when something hasn't been workshopped and it only passed through a single designer's hands in terms of the crafting of it.

But that's just my two cents,
--Neil

[Pedro Coelho]

I'm with Neil.

Besides, most NDAs will prevent "workshopping", since you're not allowed to share a publisher's work in progress with people outside the project... so I'm not sure it's a habit you'd want to cultivate.

[DM_aka_Dudemeister]

Yeah, as much as I'd love to advance further in RPGSS, I want to do it my way. Internalizing the lessons I learned from past years, past contests and new bits of game design I accrue over time.

Workshopping an RPGSS entry beyond having a fresh set of eyes for spelling, grammar, formatting and at least one person who tries to break the item is about as far as I want to go. I want my idea to shine, not become refracted by too many lenses.

This philosophy might not have garnered the votes to advance into top 32, but the fans of my items and other output on the boards have been passionate and always encouraging.

Also, I want people to like my ideas not just like me. The more people see my item, pre competition, the higher the chance of bias for my item. Conscious or otherwise, I want my idea and execution to move me through that first hurdle. Because the rest of the comp does have a personality component to it, that blind audition means a lot to me.

[Taylor Hubler]

I believe my process does address some of those concerns that have been expressed by Neil and others. By keeping the people you share your submission with down to no more than three or four you reduce the risk of creating favoritism within the voter base, or of losing your voice in the submission. By keeping that groups involvement down to just commenting you have much more control of the process. The group then can be much more like a sounding board, and not an actual workshop.

I should note that the only thing I actually workshopped was my monster. I showed my item to a couple people to get a feel for how people like it. I showed my map to a couple people because they were friends and were curious about what I was doing. I workshopped a monster because I have only made about five of them and I am not as familiar with the process.

Neil, you are right, I am going to call you a hardcore badass, but I think you proved that not in winning RPG Superstar but in your body of work afterwards. You clearly have a natural talent for the work, and I am sure that you built up and fostered that talent in your own way over the years.

You also won RPG Superstar when it was much smaller and had much fewer contestants with repeat years and 3PP credits. I am not pointing this out to lessen your win or your experience, as your submissions for the contest are still some of my favorites. I am pointing this out to express one of the many ways the contest has changed. The talent making it into the top 32 is increasingly improving and contestants are going to need to take advantage of any tools they have access to in order to compete.

I also see this contest as a great place to foster and educate new talent, and much of that new talent is expected to show competency in skills they do not yet have. A small, limited and controlled workshop is a great place to gain a crash course in those skills, be it creating a monster, making a map, or building an encounter.

While comments on my submissions are great, comments on a magic do not always translate well to a monster, and comments on a map have very limited application to anything outside of encounters. I can learn from those comments, but those comments are really only helpful in improving that limited area of the game.

While NDAs do prevent you from talking about your work, the contest does not have a strict NDA limiting a contestant's ability to share their work with a few people before submitting it. I do admit that this year has seen a much more vocal admission of using workshops, but maybe this is just another evolution of the contest.

I can see where the opponents are coming from though. It is more impressive when someone is able to make it all the way through the contest completely on their own. I can see where people may be concerned about creating voting bias. I understand that there is concern about contestants getting around Rule 5 through their friends. I believe we can find a solution that satisfies both sides of this problem.

No matter what people feel and believe, workshopping it here and it will only become more prevalent. It is up to us to shape the nature of it and help people do it right and do it in a way that benefits the contest as a whole. Complaining about it isn't going to make it go away, but we can figure out how to adapt and shape its nature.

[Curaigh]

The Raven Black wrote:

I am quite new at helping contestants through workshopping and I was met with a daunting challenge. I would like to know how other people tackle it.

One of the person I helped workshopping for the monster round is now in the Top8 and asked me for help reviewing an encounter, which I did.

I already had trouble with keeping my review of the monster unpolluted by my own design ideas. I was easily finding ways to improve the monster I was reviewing but I adamantly did not want to mix these ideas with those of the contestant.

I decided to solve this conundrum by making comments on the facts of the monster and trying as much as possible to ask questions about points I think needed to be addressed, rather than give the answers I would have given if I were in the contestant's place.

Because I just love using my imagination to fill the gaps seamlessly and it does come very easily to me. But in such an exercise, this becomes quite the hindrance.

And if that was already hard in the monster round, it has become terrifying when reviewing the encounter.

How do you, experienced workshoppers, do this ?

I don't comment in any that I have reviewed. As a matter of training myself, I look for my comments in an entry and see if they were accepted or not and what the judge & voter's reaction have been to those areas. My comments generally go toward the contest & not the entry (swing more for the fences, voters hated X entry last year) especially for the monster rounds.

Actually after 9 years, if I point something out in your item, you might as well ignore it or do the opposite. :P

A couple of year's ago one of my pit crew was fretting (privately) because so many people were missing a major component of the entry. He sent an email saying 'I could just kiss Bill' because someone finally got it. I put a smiley on his comment & the author thought that even that was too much commentary from those 'in the know' & I deleted it.

More recently a judge had a problem on an entry I reviewed. It was something that never came up in the reviewing. Because it never came up in the review (I came to the same conclusion independent of the author) the author was OK with my commenting on this particular problem (& I went no farther than that in my comments).

[Grumpus]

I agree with Neil.
To me this contest is to find a 'Superstar'.

And if you have to have multiple people look over your work and edit it for grammar, and edit it for formatting, and look at it for flavor,concept etc, then you probably aren't a superstar. You could be a very talented designer, but not a superstar.

Once you get freelance work, I think it is fine to have some eyes look over your work, but for this contest, I think it should be all solo, but there's no real way to enforce that.

[Taylor Hubler]

Grumpus wrote:

Once you get freelance work, I think it is fine to have some eyes look over your work, but for this contest, I think it should be all solo, but there's no real way to enforce that.

I disagree that contestants should be doing their work in the way you are proposing, though my comments above make that disagreement pretty clear.

You touch on a really, really important point though. There is no way to enforce any policy that limits how the contestants work on their pieces behind the scenes. I don't think there should be, but I do think that contestants should be encouraged to be more transparent in their own processes. I don't believe that will have a big effect on voting, but it could help educate future contestants and help new freelancers learn some good practices.

I have nothing against people having their own tastes and opinions on what makes someone a superstar. If they don't want to vote for people who have had coaching or a small workshop, that is there decision. I may disagree with you, but that is just one of the great things about creative endevors. There isn't one right way to do something, and there really isn't one true right way to view someone's work.

[Garrett Guillotte]

Taylor Hubler wrote:

There is no way to enforce any policy that limits how the contestants work on their pieces behind the scenes. I don't think there should be, but I do think that contestants should be encouraged to be more transparent in their own processes. I don't believe that will have a big effect on voting, but it could help educate future contestants and help new freelancers learn some good practices.

That's in part why I wish there was a round that included public collaboration with a judge that's more like real freelance work. Say if round 1 is to take the judges' feedback on an item, revise the idea, resubmit it, get more feedback, etc. It's harder to hide outside influence when the timeframe between milestones is tighter, and the public would get to see how contestants respond to criticism (and contestants could actually respond to the judge's criticism while voting is live, which would be a nice twist).

The logistics of that have to be a nightmare, though.

[Taylor Hubler]

I was thinking last night about all this and I think maybe we should be diving deeper into this issue of what is appropriate and what isn't when it comes to getting help on your submission.

There are ways of workshopping that I believe we should attempt to discourage from this contest, and we actually have much more of an ability to do so. While we cannot discourage workshopping entirely we can shape it into something much more appropriate and helpful to everyone. This is because it is pretty clear that that the people doing the coaching in workshops all run in similar circles and are loosely connected to each other.

What do I believe needs to be discouraged?

• Going into a large group like Freelance Forge, a company's group chat, or a private social media group and posting a link to the unfinished submission without consideration of who might see it or who might comment on it. You are actually taking a big risk. If you spread your submission out in such a way there is a possibility that a judge will see it. You could also create voting bias, and I am very much against actively doing that. Lastly, if you don't know who is going to look at your piece you don't know if you can trust them. Not only could their advice end up hurting your submission (sabotage at worst, unintentional mistake at best) but they could also put you in a position of toeing the line of a DQ.

• Showing your coaches an unfinished piece. My drama teacher in highschool had a saying: "Never show an idiot an unfinished piece of work." While your coaches shouldn't be idiots, if a submission is unfinished they will point out something you intend on working on or changing in the future. You also waste their time, as they will be spending it commenting on something you are already aware of. The exception to this could be the map, but that is going to depend on how you are controlling workshop and who your coaches are.

• Expecting people to do the work for you. It is okay to not know how to do something, but you shouldn't expect your coach's knowledge to make up for that. Ask them where to find the information you need and then go to the primary source. A coach should be there to get a feel for how people will react to your submission, not to fill in the template for you. They should be there to help you make minor adjustments, and not to walk you through the whole process. There are enough guides, blogs, and forum posts to teach people how to build items, monsters, and encounters, you shouldn't rely on your coach to pass that knowledge onto you.

• Allowing other people to write or work for you. This is against the contest rules. Like I pointed out above, limit your coach's ability to just comments. Do not allow anyone to have the ability to directly edit your submission.

• Encouraging people to go in and clarify something for your while voting is ongoing. This is wrong, and I very much think that this should be something that could get people disqualified if they are caught doing it. In the future I am going to encourage all of my coaches to refrain on commenting on my item. If there was a change to the rules to address workshopping, I would hope it would discourage such clarifications.
  

I have a question for those who are against workshopping: Assuming that we cannot eliminate workshopping competely, in what ways can we shape the nature of it to benefit the contest and make it more acceptable to a wider range of voters? Did I miss anything above in my own suggestion?

[Neil Spicer]

Taylor Hubler wrote:

I have a question for those who are against workshopping: Assuming that we cannot eliminate workshopping competely, in what ways can we shape the nature of it to benefit the contest and make it more acceptable to a wider range of voters? Did I miss anything above in my own suggestion?

Let me make something clear about my position. I'm not suggesting that workshopping as a means of improving yourself as a designer is wrong. What I'm suggesting is that workshopping your actual round-by-round submissions is wrong. Additionally, that's only my opinion as a former-contestant and a guest judge, not an actual Paizo employee.

I think all the insights and educational benefits you're seeking from a workshop can be done outside the competition to improve yourself rather than any specific design/submission for RPG Superstar. You've got all year-long to do that as much as you want. But, during the competition, I think you're designing it by committee if you're sharing your work for the purposes of soliciting input and suggestions on how to improve it beyond what your own imagination and learned skills can provide you.

But, as always, that's just my two cents,
--Neil

[frank gori]

I truly believe there's a generational shift coming when it comes to this issue in not only the gaming community but content creation in general. Technology has made it easier than ever to collaborate and share ideas. While I believe true collaborative efforts cross the line in this contest (and we have a no line editing rule, you can only suggest with my crew), I also believe that this conversation in retrospect will look dated and silly in about 5-10 years.

Take one of my favorite websites, Cracked.com. They have a forums for content creators that basically work their articles sometimes very quickly sometimes over the course of months refining the content until it becomes public. One fo the reasons cracked articles are so good and have so few editing errors is because a few dozen eyes have poured over that sucker before you ever saw it.

Kobold Press had(has?) something similar though I haven't been involved there in a couple years so that might have changed. Flying Pincushion Games, Freelance Forge, Four Horsemen, d20pfsrd games and for the most part about half a dozen or so companies I'm familiar with do something similar.

I truly think it is going to eventually be the dominant way content gets created. If a publisher wants things done old school I do it that way but eventually those companies will be seen like we see companies that insist on using fax machines, relics.

Edit: FYI I'm enjoying this discussion greatly as I have a passion about the subject.

Again with relation to the contest, as long as you're the one to carry out suggestions and you're not copying someone's idea word for word but making the suggestion your own I would not actually call that design by committee. Hell if the definition is that loose then people who adapt ideas from other sources including mythological ones are designing by committee. I think it was Jim Butcher's blog that said something like stealing form a modern source is plagerism, stealing from mythological sources and fairy tales is called "research."

[R Pickard]

I think there's a semantic problem here as regards to the meaning of "workshopping." I think different people have a different sense of what it means.

I think most agree having some folks look over your submission before you submit it is a good idea. Gathering feedback and technical corrections is good ("this needs some oomph"; "you spelled this wrong"; "I don't think this monster you reference exists."). My sense from Frank's post and elsewhere is that's what he and others are trying to talk about.

"Workshopping" CAN indeed however mean deeper input---writing or design by committee, which is where I think Neil is concerned. If someone starts feeding specific suggestions like, "This should be able to fart rainbows and shoot lightning out of its eyes," and you add that to your entry, and then even perhaps advance because people liked "your" rainbow-farting ability so much, you essentially advanced due to someone else's creativity than your own. When stuff like that happens, we aren't seeing what YOU, as a designer, by yourself, are capable of, and that can cause problems down the line, both in terms of legality as well as relying on an individual to provide quality work (amid a myriad of other issues, if you lost touch with rainbow-fart guy and no longer have him to get ideas from, and can't do the same on your own, you can't deliver what others have come to expect from you).

From my POV, the take-home is that giving specific feedback is good, but you need to give feedback on WHAT IS THERE, not what you would make of it (because it's not your entry).

Development of course can be and usually is collaborative, but this is still a contest that assesses an individual's skill---just as in this industry, assessing individual skill is still crucial to determine what any given person may contribute TO a team.

[frank gori]

Death Quaker spots the translation error methinks...

[Taylor Hubler]

Neil Spicer wrote:

Taylor Hubler wrote:

I have a question for those who are against workshopping: Assuming that we cannot eliminate workshopping competely, in what ways can we shape the nature of it to benefit the contest and make it more acceptable to a wider range of voters? Did I miss anything above in my own suggestion?

Let me make something clear about my position. I'm not suggesting that workshopping as a means of improving yourself as a designer is wrong. What I'm suggesting is that workshopping your actual round-by-round submissions is wrong. Additionally, that's only my opinion as a former-contestant and a guest judge, not an actual Paizo employee.

I think all the insights and educational benefits you're seeking from a workshop can be done outside the competition to improve yourself rather than any specific design/submission for RPG Superstar. You've got all year-long to do that as much as you want. But, during the competition, I think you're designing it by committee if you're sharing your work for the purposes of soliciting input and suggestions on how to improve it beyond what your own imagination and learned skills can provide you.

But, as always, that's just my two cents,
--Neil

R Pickard hit the nail on the head: we have a translation issue.

Neil: I have been talking about "workshopping" within the contest the entire time, and my efforts have been to try and shape the conversation towards how to encourage people to get feedback they want without moving into the development-by-committee we both agree is wrong.

Let me be clearer here. I do not endorse a development by committee inside the contest and I avoid the practice outside of it as well.

Maybe we need new vocabulary. I started using the term coach and coaching above. Instead of workshop and workshopping, would it be better to call what I am proposing as coaching? The few people you are showing a submission to are not supposed to do development work, but act as coaches that can give you a sense of how voters will view your piece. It isn't their job to give detailed development comments but to point out trouble areas, let you know what isn't standing up well on its own, and what is really working well.

Contestants involved with RPG Superstar are going to want to show people their submissions before they are turned over to Paizo to get some form of feedback. People are going to want to be there to lend support and coaching to the contestants. I agree that there can be problematic practices related to this. I want to explore the ways we can, as a community, make this work for the contest.

So my question still stands (with some revision):
Assuming that we cannot eliminate coaching from the contest completely, in what ways can we shape the nature of it to benefit the contest and make it more acceptable to a wider range of voters? Did I miss anything above in my own suggestion?

[Jacob W. Michaels]

Honestly, at the risk of being blunt, my feeling is if someone has to resort to group designing, they're probably not going to be good enough to advance in the contest anyway. No amount of workshopping is going to make up for someone who's lacking the necessary skills.

Not only that, maybe I'm being naive but I think anyone who's really serious about winning this contest wants their entries to be their own work, reflective of what they can do on a daily basis. It does me no good to have a dozen people design your magic item if you're not going to be able to have them do all the work you end up getting offered afterward. If you don't have the skills, the publishers are going to realize it quickly and stop offering you work.

[Pedro Coelho]

I can only speak for myself, but regardless of terminology, I had no problem understanding what you were referring to as "workshopping" (now rephrased as "coaching"). Personally, I don't endorse it; it is an individual competition, after all, not a team one.

It's a harmless offense when you're talking about your ideas to one best friend, or maybe your partner, outside the confines of the contest, but when a number of "coaching" circles begin to crop up in the forums, teaming up contestants with vocal and engaged members of the voting community, that starts to raise some questions about sportsmanship.

A way to make it more acceptable, to me, at least, is: don't do it. It's hard to draw a line at where harmless feedback ends and where design by committee begins, so just don't go there. To me, objectively, the line is: if you're getting feedback from two people outside the Paizo community, and that feedback is on your idea's staying power or on the clarity of your text, that's ok, but that will of course vary from person to person. I don't think anything outside the parameters I've stated can make "coaching" more acceptable, personally.

Of course, like it was said before, it's impossible to enforce this, but we've all played paladins and know how the honor system works, right? It's the next best thing we have.

One other thing that kind of puts me off, frankly (and this one is on the voting community), is how many offers to review the contestants' work in progress are being thrown around. It's puzzling to me. After all, are we not going to see their entries as soon as they're published, so that we can offer our feedback then, during the voting window? What's with the rush?

To the contestants, just trust your own talent and hard work, follow the feedback between rounds (and between professional assignments, once you get there), and apply what you learn to your future pieces—by yourself. You'll be glad you did.

But that's just my worthless penny—I don't have two cents to actually know what I'm talking about. ;)

[Nick Wasko]

Like Frank, I think this is a fascinating discussion and I'm glad people from both sides are bringing their opinions and rationales to the table.

It seems to me that any future changes concerning coaching - whether in the form of contest rules or general attitude towards the practice - would best benefit the contest if they reflect the purpose of the contest itself. From my perspective, the goals of RPGSS are:

1) Engage the Paizo community in a fun event.
2) Produce an awesome module voted for by the aforementioned community.
3) Drum up new game design talent that can produce high quality material according to (but not limited to) Paizo's standards.

Coaching could potentially help or harm point 1. On one hand, people who turn to their gaming friends, loved ones, etc. for feedback or voting support bring new viewers to the Paizo messageboards, which increases interest, public exposure, and possibly new contributions to the community. I got involved on the Paizo messageboards (and subsequently PFS organized play) as a direct consequence of Chris making it into RPGSS 2013 and asking me to proofread his entries. On the other hand, if new people feel like they are fighting against established "cliques" who all critique each other's work, it could make them feel isolated and undermine the goal of engaging new members.

Point 2 probably doesn't change much regardless of coaching, since the public votes on the module and Paizo staff play a key role in making the final product shine. That being said, it still derives value from point 1.

Point 3 seems like the main elephant in the room. A few people here have voiced concerns that coaching disguises the true quality of the contestants. Since solo and communicative design work both exist within the industry and neither is necessarily a "right" or "wrong" way to make a product, I think guidance on this point will have to come from Paizo itself. The contest rules already forbid entries that have circulated through public forums or received public feedback, but if Paizo holds freelancers to higher standards as the norm, the contest regulations should reflect their default assumptions. I've never written for Paizo, so I don't know how frequently they use NDAs or how much they limit editing or proofreading to in-house staff, but as a contestant I'd like to think the resources I'm allowed to access during the contest would also be available to me when working on commissioned assignments. If that is not the case, then tighter restrictions on feedback, even if they are difficult/impossible to enforce, would be a service to those competing for the writing contract. At least that way they know what they are getting into when they submit for the open call.

[Oceanshieldwolf]

Frank Gori wrote:

Again with relation to the contest, as long as you're the one to carry out suggestions and you're not copying someone's idea word for word but making the suggestion your own I would not actually call that design by committee.

I'm not agreeing with you here Frank. Essentially I would call that design by another person's creativity. A la rainbow fart guy/gal.

--------------------------------------------------------------------------- ---------------------

* Previous to the conversation with RFG you had NO rainbow farts.

* After conversation with RFG your entry now HAS rainbow farts.

* Rainbow farts are this year's reverse filigree - unique, eye-catching and all the rage.

--------------------------------------------------------------------------- ----------------------

***Understand this is just in the interests of RPGSS.***

As those who know me from Kobold Quarterly/Kobold Press patronage projects days (Journeys to the West, Midgard Legends, Midgard Tales) or the legion Multiclass Archetypes threads, I'm am an idea shotgun - I blast out concepts based on what you show me, and sometimes something sticks - if that is what you call the "bounce" then that is what I excel at. And I agree that works for publishing ventures and group design projects.

But this competition is about the sole entrants creativity, and much less about how well they can communicate in a workshop group environment. Perhaps you are right, and in 5-10 years this conversation will look silly, but only if RPGSS has a design-by-committee/workshop round.

[Jeff Harris 982]

I had tried to keep silent on this topic, but seems I just can't.

We have already hashed out that there is a difference between Coaching and Design by Comity.

So, what is the main issue for me, it is thus:

There is a certain level of restrictive (we are not there yet) that turns the competition from fun to paranoid and community damaging IMO. Should we come to a point where the official rules clearly state (as I am aware they do not state this yet) "You may not share your item with anyone and doing so is tantamount to cheating as we therefore assume your are designing by comity and NOT as a singular designer." I dare say at that time we would see the end of RPGSS as we know it, and the start of a different contest which I suspect would have a very different feel to it.

Of course, please be aware I do not support any of the following: voting cabals, design by comity control *(that is, the item designer allows others to make changes to their item, or creates an item from comity input vs. personal choice), or any other practice that removes the designer from the role of main author of their RPGSS item, map, monster, or encounter.

I do however fully support allowing RPGSS contest designers to seek feedback and feelings on their item or work from those they chose to share their work or its concept with.

I of course do not know how Paizo does things in house per say, but I would assume no one is an island unto themselves at the office. Thus, why is the "island unto myself" as a designer valued in RPGSS? I freely admit I shared my item with certain folk I trust the judgement of, and respect the feedback from.

However, nothing, said by any of these folks has or will ever force me to change my item, nor would I ever simply write into my item the words of another person, no matter how trusted. I am always the final arbiter of my work, and I do feel that it is somewhat personally degrading to imply that seeking feedback or advice from others makes me a less capable designer for doing so.

I see such as a chance to get a perspective on my item or work that is not my own, and I feel it is important to do so for a contest of this nature, as the voters are unique individuals, likely do not share some or all of my same gaming preferences, and may very well see my item or work from a completely different angle than I do. And the only way to account for this is to share your item or work in some capacity, be it in full or just as a discussion of concept or mechanics.

But, as others have said, and just like them, this is just my two coppers, and my experiences are just that, mine, and of course will vary greatly from person to person.

[Taylor Hubler]

This brings us to questions that only Owen and other Paizo employees can answer. I am sure Neil or other people who have written for Paizo could also answer most of these.

What expectations does Paizo have of contestants, winners, and freelancers? Are those expectations different from each other?

Are those writing for Paizo able to get feedback before a deadline, and if so what kind of feedback are they allowed? Could I go to my regular gaming group and playtest something before I turn it in? Could I ask the developer I am working with to give some feedback if I am unsure about something I am working on?

What are Paizo's feelings about contestants going to other people for coaching, feedback, or advice?

[Anthony Adam]

Coaching...

I took part in SKR's online design courses he runs - between competitions.

Feedback...

I take it on the chin and sometimes right in the g'nads but keep coming back for more - am I mental?

Advice...

I have been both pit crew member and asked for general feelings, and I have helped others -

I tend to keep to specifics - that spell is the wrong aura, your template is broken, your lore is not correct, etc

I will say things like, I am confused by this, or I like this, or this is really good, or you have passive writing in there but actively avoid suggesting design changes as much as possible.

The item is your design, and if you win, you are going to have to do the work yourself as most pit crew members disappear to practice their own skills for next time once the competition is done.

[Garrick Williams]

When designing, I typically debate with myself between two or more approaches. So, I create a draft of each approach and then get my pit crew's impressions of it. Based on how they felt about them, I make a decision. For example, let's say I can't decide between designs A, B, or C. So I write a draft for each of them and ask which one they like the best. This can make me think of a new approach that uses a combination of the existing ones based on their reactions.

However, I gain the biggest benefit from merely forcing myself to create concrete drafts of multiple approaches. This alone often shows me which design choices might work and which ones aren't viable. I think this is the most important use of your pit crew -- making sure you always have something to show for the time you spend. Navel-gazing rarely gets you very far, especially when you have a deadline.

[Captain Phoenix]

Certainly many of you have more experience than I do, but I do think that design, by definition, is an iterative process that requires “feedback”. That feedback takes many forms, such as testing, gathering of opinions and research, which is fed back into the design. Managing this feedback effectively, while maintaining their “personal voice”, is a critical skill for a good designer. After all, we’re not designing in a vacuum; we’re designing for an audience whose opinion is important.

Feedback is already such an integral part of the contest; RPG SS would be nothing without the judges giving their feedback after each round. If feedback is really hurting the process, the contest or masking the true ability of the contestants, then that’s going to come to light at some point. Rather, I think that whatever feedback contestants are receiving from their pit-crews they’re assimilating and it’s making them better designers overall.

[frank gori]

Captain Phoenix wrote:

Managing this feedback effectively, while maintaining their “personal voice”, is a critical skill for a good designer. After all, we’re not designing in a vacuum; we’re designing for an audience whose opinion is important.

^THIS^

+1000

You would not believe how many times I've gotten contradictory feedback on an item or archetype. Advice or feedback is just a flashlight on the parts of your design that need attention, not a cheat code for an easy win.

[Mikko Kallio]

frank gori wrote:

Captain Phoenix wrote:

Managing this feedback effectively, while maintaining their “personal voice”, is a critical skill for a good designer. After all, we’re not designing in a vacuum; we’re designing for an audience whose opinion is important.

^THIS^

+1000

You would not believe how many times I've gotten contradictory feedback on an item or archetype. Advice or feedback is just a flashlight on the parts of your design that need attention, not a cheat code for an easy win.

Very much agreed. I was thinking the same thing while reading some of the earlier responses, and I couldn't have said it better than Captain Phoenix or Frank did. Just a flashlight, not a cheat code. (Unless someone else actually writes your entries, but that's not what we're talking about here.)

[Taylor Hubler]

Triple agreed.

I may have gone on a bit and been unclear, but my intention with advice I gave was to increase the power of the flashlight while minimalizing the possibility of cheat code.

[RJGrady]

Is it possible that the Superstar format will tend to recruit creaky, stubborn creative types who are enamored of their own personal vision? I certainly hope so!

[Kerney]

RJGrady wrote:

Is it possible that the Superstar format will tend to recruit creaky, stubborn creative types who are enamored of their own personal vision? I certainly hope so!

I can say this, the items I've entered which have received the least positive reaction have been the one I've most been enamored with and have been most stubborn about getting just right.

I know one semi pro who I've known with who has this attitude doesn't seem to work for others and I don't think will be judging this contest.

Basically, you need to be able to realise when your darlings are only your darlings, get some feedback people you trust, and play well with others.

[RJGrady]

Well, not every idea's a winner.