My Review Process


RPG Superstar™ 2008 General Discussion

Legendary Games, Necromancer Games

OK, I wanted to share my process for reviewing these submissions.

I quickly scanned the submissions and realized every one of them is awesome. This wasnt going to be a case where I could easily divide one from the other based on some truly objective criteria. I was in a bit of a quandry, because I believe that I should be as objective as possible. I didnt want to descend into pure subjectivity. But as I said there was not a ton of objective stuff here that seperated these submissions (there were some things). I really wanted to provide comments taht would be helpful to the voters.

So I thought and thought. And this is what I came up with.

I am a professional publisher of adventures. I have read TONS of adventure submissions. Rather than grade each submission, which seemed cheesy since so much is subjective, I hit on this solution: I would provide a kind of real time gestalt reaction to each submission as I read it, reflecting the exact thoughts I would have if I was simply reading one of the many emailed adventure proposals that I have reviewed over the years. That way you can see the thought process (deranged, admittedly) that an ACTUAL publisher goes through as he reads and actual submission.

So instead of looking at these as submissions in a contest, I looked at them as actual submissions as if they were submitted to me at my company for publication as a 32 page module. And I literally do it from the title to the end, with my thoughts as I go.

Why? Because I thought that might be an interesting perspective for the voters. Plus, I dont like dressing up totally subjective impressions with a grade. As if there is some universal right way to do things. Prior rounds lent themselves to some more objective review. These seem much more subjective to me. So I gave you the ultimate in subjectivity--my actual line by line thoughts.

I hope you find it useful. In them, I reflect on what works and doesnt work for me in submissions for adventure proposals. I try to bring out some things that I think are universally good or bad, but I dont claim to know it all or to believe it applies the same to all publishers.

I also felt that, simple as it sounds, I am a publisher and have done this before many, many times. I cant pretend to be a genius at item design or monster design. But I sure can tell you what makes me want to greenlight an adventure submissions. So I figured I would give it to you as raw and direct as possible. So I did. I told you when they lost me or when I got all giddy. I told you when I cheated and read ahead and when I got bored.

I hope you guys find it entertaining. And helpful. At least a little anyway.

Clark

Dark Archive Contributor, RPG Superstar 2008 Top 4

Clark, your honesty has consistently been one of the most wonderful things about this AMAZING, life-changing competition.

As always, I look forward to your insights, your comments and your criticism.

Legendary Games, Necromancer Games

Boomer, you were one of my top 2. :)

I really cant wait to see what the voters do with these submissions.

Bottom line: even the bottom 2 are clearly publishable. Only under the incredible scrutiny of RPG Superstar would the phrase "bottom 2" be applied to any of these 4.

Clark

The Exchange

Maybe Paizo should publish them all, if they are that good.

The Exchange RPG Superstar 2011 Top 32 , Star Voter Season 8 aka Sect

Aubrey the Malformed wrote:
Maybe Paizo should publish them all, if they are that good.

You know, I remember the first season of Spike's "Ultimate Fighter", where the final two fighters performed so well, UFC ended up offering both of them contracts...

Sovereign Court aka Robert G. McCreary

I've always been impressed with your comments on previous rounds, Clark, so I'm definitely looking forward to see what you have to say this time around. But reading about your review process has made me even more impatient to read those comments NOW! :)

Aubrey the Malformed wrote:
Maybe Paizo should publish them all, if they are that good.

Sounds like a good idea to me! ;)

Scarab Sages Contributor, RPG Superstar 2008 Top 4, Legendary Games

SargonX wrote:

I've always been impressed with your comments on previous rounds, Clark, so I'm definitely looking forward to see what you have to say this time around. But reading about your review process has made me even more impatient to read those comments NOW! :)

Aubrey the Malformed wrote:
Maybe Paizo should publish them all, if they are that good.
Sounds like a good idea to me! ;)

Ditto on all counts! I think the stream-of-consciousness review will be very interesting, and I have enjoyed the brutal frankness of comments (both good and bad... [cough]demon horse[cough]), from objective specifics to subjective 'feel' (or 'tilt' in Clarkese).

I have enjoyed reading all the judges' comments and each has a different flavor, but I want to give extra props to Clark for going the extra mile in being very active in further responses in each thread beyond the initial judging section.

Noon can't come soon enough!

Paizo Employee Chief Technical Officer

Clark Peterson wrote:
I hope you guys find it entertaining. And helpful. At least a little anyway.

I, for one, really, really liked your approach here.

Dark Archive

Clark Peterson wrote:

So I gave you the ultimate in subjectivity--my actual line by line thoughts.

I hope you find it useful.

Occasionally, to sate my inner dork, I like to read online 'directors cuts' of people's fiction, where the author goes through something they've written and offer their own behind-the-scenes thoughts and different 'takes' that they ended up scrapping before settling on a particular scene or description. I kinda love this style, even if it's exactly the sort of thing the writer can't do in actual writing (since they aren't gonna be holding your hand and explaining what they meant when you read their stuff, it has to be able to stand on its own).

So yeah, 'hearing' your reactions in real-time is neat, and I'm sure it's very interesting for the writers to hear what particular choices they made were big hits and which ones nearly turned into critical misses...


Clark Peterson wrote:
That way you can see the thought process (deranged, admittedly) that an ACTUAL publisher goes through as he reads and actual submission.

Incidentally, you get way more feedback here at RPG Superstar than you do when someone answers a query. Having not one, not two, but three industry greats picking apart your stuff and a whole bunch of amateurs, hangers-on, ne'er-do-wells (me! :D ), and the occasional insightful guy off the street is a real boon to a writer.

Community / Forums / Archive / Paizo / RPG Superstar™ / Previous Contests / RPG Superstar™ 2008 / General Discussion / My Review Process All Messageboards

Want to post a reply? Sign in.
Recent threads in General Discussion
Losing items