In previous years some of the judges were willing to give input on items that did not make it in the top 32 if the person posted their item in a specific thread.
Are any of the judges willing to do this again this year?
I thought it was very enlightening reading about the thought processes of the judges on why the item did not make it in their top 32.
There's been some discussion about that and I believe Sean has something in mind. Just be patient and keep checking back here.
I'm going to see what sort of data-dump Ross and Gary can do with all the threads in the judges area, so I can just look up an item, see what I said before, and paste that, rather than having to re-analyze hundreds of entries (which is usually what causes my participation to fall off as later rounds' judging occurs...).
I'm going to see what sort of data-dump Ross and Gary can do with all the threads in the judges area, so I can just look up an item, see what I said before, and paste that, rather than having to re-analyze hundreds of entries (which is usually what causes my participation to fall off as later rounds' judging occurs...).
Good luck Sean, I know we all will appreciate anything you can do but after hearing about the turn out this year you may have your hands full even with Ross and Gary's help.
Star Voter Season 6, Star Voter Season 7, Dedicated Voter Season 8
|
These are the people that made what is almost definitely the best tabletop fantasy RPG to date. I have no doubts they will rock the judges table with ease.
Never the less, good luck gentlemen!
(Admittedly I just think it's awesome that game developers are replying to my post. You guys are living the dream.)
There are pros and cons to the process, which has been much discussed in prior years.
First, people need to ask to have the comments about their item that didnt make the cut. Why? Because while all entries are owned by Paizo, they were also all submitted anonymously with the understanding that the judges will review them privately and they will either make the top 32 or not. Creating anything is a very vulnerable process and some people won't submit to open criticism. So making all submissions open is actually a negative. If people want critiques, they in the past have usually gotten them--as well as can be done.
Second, judges judge in shorthand and with some brutal frankness that many contestants won't appreciate. And since we are "sorting" each item quickly because of the sheer number, we arent holding each submission and treating it as a delicate flower. What follows, as a result, can often seen harsh if the judge's comments are simply revealed. Given that we are (or in teh past I was) quickly sorting items, judges often focus on the negatives or the flaws of the item--the reasons to reject it--rather than its positives. So it is entirely better to have someone request critique and then have the judges post "summarized" comments from the original judge's forum rather than just directly posting the judges comments. You can see this from the prior "feedback" threads where I summarized what some of the issues were. But in doing so I also balanced out some of the positives of the item, which often were glossed over in the comments.
I hope that helps people understand the process. I believe in transparency and I think this is as transparent a contest as you will ever find, with as great of a community as you will ever find.
|
There are pros and cons to the process, which has been much discussed in prior years.
First, people need to ask to have the comments about their item that didnt make the cut. Why? Because while all entries are owned by Paizo, they were also all submitted anonymously with the understanding that the judges will review them privately and they will either make the top 32 or not. Creating anything is a very vulnerable process and some people won't submit to open criticism. So making all submissions open is actually a negative. If people want critiques, they in the past have usually gotten them--as well as can be done.
Second, judges judge in shorthand and with some brutal frankness that many contestants won't appreciate. And since we are "sorting" each item quickly because of the sheer number, we arent holding each submission and treating it as a delicate flower. What follows, as a result, can often seen harsh if the judge's comments are simply revealed. Given that we are (or in teh past I was) quickly sorting items, judges often focus on the negatives or the flaws of the item--the reasons to reject it--rather than its positives. So it is entirely better to have someone request critique and then have the judges post "summarized" comments from the original judge's forum rather than just directly posting the judges comments. You can see this from the prior "feedback" threads where I summarized what some of the issues were. But in doing so I also balanced out some of the positives of the item, which often were glossed over in the comments.
I hope that helps people understand the process. I believe in transparency and I think this is as transparent a contest as you will ever find, with as great of a community as you will ever find.
+1.
By the way, I am certain that the feedback this year will crush anything I ever did. Sean is a genius. And I dont know if I have ever seen anyone do as thorough and detailed of an analysis as Neil does on items. I was just re-reading last year's feedback thread and his 5000 word comments are just crazy. Plus you add Ryan Dancey to that. You guys have some real heavy weights reviewing your entries. I have tons of passion, true. But I was never the match for Erik or Wolf or Sean in the mechanics departments. You guys are in for a real treat this year.
|
|
By the way, I am certain that the feedback this year will crush anything I ever did.
Don't sell yourself short. You were honest, and yet also often very 'gentle' in your criticisms, which was a nice touch, and may have spoiled us a bit, given that not everyone has that touch.
And I dont know if I have ever seen anyone do as thorough and detailed of an analysis as Neil does on items.
He analyzes the hell out of stuff, which is awesome. His brain is like an engine.
Director of Brand Strategy
|
By the way, I am certain that the feedback this year will crush anything I ever did. Sean is a genius. And I dont know if I have ever seen anyone do as thorough and detailed of an analysis as Neil does on items. I was just re-reading last year's feedback thread and his 5000 word comments are just crazy. Plus you add Ryan Dancey to that. You guys have some real heavy weights reviewing your entries. I have tons of passion, true. But I was never the match for Erik or Wolf or Sean in the mechanics departments. You guys are in for a real treat this year.
You left a name off that list, Clark... :-(
RPG Superstar 2012 Top 32
,
Champion Voter Season 6, Champion Voter Season 7, Champion Voter Season 8, Champion Voter Season 9
|
You left a name off that list, Clark... :-(
This was my first year competing and I read mostly last years feedback on the items. I liked Clark's responses cause they were smaller and did not confuse my small ferret-like brain processing ability. OOH, a shiny...
LOL, Snorter, reminds me of the WOW episode of Southpark, with Cartman sitting in the basement at his desk yelling for his mother...
If you have seen it then you know what I am talking about and if you haven't then you really should see that one episode.
|
By the way, I am certain that the feedback this year will crush anything I ever did. Sean is a genius. And I dont know if I have ever seen anyone do as thorough and detailed of an analysis as Neil does on items. I was just re-reading last year's feedback thread and his 5000 word comments are just crazy. Plus you add Ryan Dancey to that. You guys have some real heavy weights reviewing your entries. I have tons of passion, true. But I was never the match for Erik or Wolf or Sean in the mechanics departments. You guys are in for a real treat this year.You left a name off that list, Clark... :-(
It's because you're The Quiet One, Mark. You're like the George Harrison of RPG Superstar judges. :)
(Which is not a bad thing at all...)
|
By the way, I am certain that the feedback this year will crush anything I ever did. Sean is a genius. And I dont know if I have ever seen anyone do as thorough and detailed of an analysis as Neil does on items. I was just re-reading last year's feedback thread and his 5000 word comments are just crazy. Plus you add Ryan Dancey to that. You guys have some real heavy weights reviewing your entries. I have tons of passion, true. But I was never the match for Erik or Wolf or Sean in the mechanics departments. You guys are in for a real treat this year.You left a name off that list, Clark... :-(
Dude. You're a Small Mite, and right next to you is the mothertruckin' Huge ORCUS, Demon Prince of Undeath, grasping firmly his Wand and exhaling pure annihilation. What did you expect ? Be lucky he didn't notice you, or there would be a mite-ly smackdown.
Sorry Mark...
"...and Mark!"
There :)
"... the Professor and Mary Ann, here on Gilligan's Isle!"
I'm going to see what sort of data-dump Ross and Gary can do with all the threads in the judges area, so I can just look up an item, see what I said before, and paste that, rather than having to re-analyze hundreds of entries (which is usually what causes my participation to fall off as later rounds' judging occurs...).
Maybe they could add a feature to "opt in"? That is, if you check the "Yes, I would like to see my submission and the feedback on my submission publicly posted" box, the "data-dump" would then pick up the "thread of comments" on your submission (after the top 32 are announced) and automagically make it available for public viewing.
The only issue with that, I can think of, is that some judges may not be comfortable with their comments going public. Adding a way to filter their comments would probably be too complicated. In which case, Sean's method of "copy-paste" would probably be the only good solution.
Just my thoughts.
Ken
The only issue with that, I can think of, is that some judges may not be comfortable with their comments going public.
Considering that some of my comments include, "this item's name is crap," yes.... ;)
It's definitely a lot easier/faster to judge a contest wherein your comments for other judges aren't meant for public consumption.
Personally I'd love if I could just check a box that yes, I'd like to see my unadulterated rejected item comments after the top 32 are announced and yes, I promise to not get offended that Sean wrote, "An item this crappy should at least promise to kill Hitler one or more times." Given time and effort demands on the judges, though, I think the current setup in which they're not released but people who are really curious can request feedback is probably the best compromise.
| 1 person marked this as a favorite. |
Maybe they could add a feature to "opt in"? ...
Just my thoughts.Ken
I think they should charge $1 for the option. :) That way Vic's time spent cleaning up the judge's comments can at least be seen as work.
Maybe they could add a feature to "opt in"? ...
Just my thoughts.Ken
I think they should charge $1 for the option. :) That way Vic's time spent cleaning up the judge's comments can at least be seen as work.
** spoiler omitted **
LOL, funny.
Quick question about your name....Curaigh....Isn't that gaelic for "Skull"?
The only issue with that, I can think of, is that some judges may not be comfortable with their comments going public.Considering that some of my comments include, "this item's name is crap," yes.... ;)
Just because I submitted the Crap Helmet, doesn't mean you have to tell everyone!! (joke of course).
Gosh....I hope noone really did submit a Crap Helmet :O
Crap
Aura faint necromancy, faint transmutation; CL 0th
Slot -;Price 1cp;Weight (Varies)
This brown substance is slightly amorphous, changing its shape to acommodate a multitude of surfaces. 1/Month, Crap can be used to improve the ghrowth of plants by 10%. Afterwards, Crap is used up.
Requirements The ability to eat food;Cost 1sp
If the judges knew their unedited comments would be viewable, even just by the items submitter, they would take more time to comment, slowing the whole process down. If you don't make the cut and want feedback on your item I'm sure there will be a Feedback thread started and the judges will give constructive criticism. Just have a little patience with them, and post as soon after the Round 2 announcement as possible.
Having my submission getting the Don Rickles treatment by the internet just sounds like fun after reading the thread. If you enter what is essentially a writing contest and cannot take criticism you have already lost.
RPG Superstar 2009 Top 32, 2010 Top 8
,
Star Voter Season 6
|
The only issue with that, I can think of, is that some judges may not be comfortable with their comments going public.Considering that some of my comments include, "this item's name is crap," yes.... ;)
Ok Wikileaks, turn your servers on Paizo ;-)
If the judges knew their unedited comments would be viewable, even just by the items submitter, they would take more time to comment, slowing the whole process down. If you don't make the cut and want feedback on your item I'm sure there will be a Feedback thread started and the judges will give constructive criticism. Just have a little patience with them, and post as soon after the Round 2 announcement as possible.
I agree. And, there is even a chance they wouldn't give that honest of a comment. They might just say..."reject"!
Obviously the current system works. Heck, it's cool that any of them even take the time to share comments on those items. That's a lot of work :)
Ken
Quick question about your name....Curaigh....Isn't that gaelic for "Skull"?
If so, I did a terrible job translating. :) It is sort of gaelic, but without the proper accent marks. curadh is how I think most translators spell it now, but the book I used said sounds like /ig/ and so I spelled it that way. :)
I agree. And, there is even a chance they wouldn't give that honest of a comment. They might just say..."reject"!
Some of the judges' comments go exactly like that.
Not mine, mind you. I have this tendency to spend 5,000 words per post. But, even on a few of them, some other judge got there before me. I'd read their assessment and just say, "Agreed. Vote to Reject." And then push the Reject button for us. In fact, I daresay I was the primary button-pusher this year. If you went to the Keep or Reject pile, odds are I pressed the shiny, red button that put you there. Okay. It's not really a shiny, red button...
Quick question about your name....Curaigh....Isn't that gaelic for "Skull"?If so, I did a terrible job translating. :) It is sort of gaelic, but without the proper accent marks. curadh is how I think most translators spell it now, but the book I used said sounds like /ig/ and so I spelled it that way. :)
So the correct translation is "Champion"?
|
|
I agree. And, there is even a chance they wouldn't give that honest of a comment. They might just say..."reject"!Some of the judges' comments go exactly like that.
Not mine, mind you. I have this tendency to spend 5,000 words per post. But, even on a few of them, some other judge got there before me. I'd read their assessment and just say, "Agreed. Vote to Reject." And then push the Reject button for us. In fact, I daresay I was the primary button-pusher this year. If you went to the Keep or Reject pile, odds are I pressed the shiny, red button that put you there. Okay. It's not really a shiny, red button...
Every mastermind needs a shiny red button. :)
Not mine, mind you. I have this tendency to spend 5,000 words per post. But, even on a few of them, some other judge got there before me. I'd read their assessment and just say, "Agreed. Vote to Reject." And then push the Reject button for us. In fact, I daresay I was the primary button-pusher this year. If you went to the Keep or Reject pile, odds are I pressed the shiny, red button that put you there. Okay. It's not really a shiny, red button...
The buttons are labelled "Cake" and "Death" aren't they?
I agree. And, there is even a chance they wouldn't give that honest of a comment. They might just say..."reject"!Some of the judges' comments go exactly like that.
Not mine, mind you. I have this tendency to spend 5,000 words per post. But, even on a few of them, some other judge got there before me. I'd read their assessment and just say, "Agreed. Vote to Reject." And then push the Reject button for us. In fact, I daresay I was the primary button-pusher this year. If you went to the Keep or Reject pile, odds are I pressed the shiny, red button that put you there. Okay. It's not really a shiny, red button...
Hey, wait a minute!
Are the keep and reject buttons real close together? Did you look really carefully before you pressed the button?? I demand a recount!!!
Not only are the buttons close to each other, they're side-by-side! And, perhaps worse...the Keep button is squarely in between Reject...and Delete! So, yeah. Hope you made it to the right pile. o_O
Heh. And, if anyone's worried their item may have inadvertantly been sorted to the wrong place...
That ought to solve the problem. :->
--Neil
Not only are the buttons close to each other, they're side-by-side! And, perhaps worse...the Keep button is squarely in between Reject...and Delete! So, yeah. Hope you made it to the right pile. o_O
Heh. And, if anyone's worried their item may have inadvertantly been sorted to the wrong place...
** spoiler omitted **
That ought to solve the problem. :->--Neil
Cough, cough... Oy.
There will come a time when I won't fall for that. Thank Apsu I have some resistance to things of that nature. ;)
Not only are the buttons close to each other, they're side-by-side! And, perhaps worse...the Keep button is squarely in between Reject...and Delete! So, yeah. Hope you made it to the right pile. o_O
Heh. And, if anyone's worried their item may have inadvertantly been sorted to the wrong place...
** spoiler omitted **
That ought to solve the problem. :->--Neil
Thankfully I've employed a fire mephit to click my spoiler tags for me.
I didn't tell him that explosive runes actually deal force damage, but on the flip side he shouldn't be hard to replace.
Muhahahaha, my pocket rogue disabled that one for me. God I am getting to really love that little guy.
Anyway back on topic, Neil, you sure it was a "shiny red button"? Oh, my bubble, cough, cough, cough, sorry went down wrong pipe...
Anyway like I was saying, oh SHINY, where were we? Sorry guys going to cut this short I keep, oh look a moving red dot.....
It's definitely a lot easier/faster to judge a contest wherein your comments for other judges aren't meant for public consumption.
Personally I'd love if I could just check a box that yes, I'd like to see my unadulterated rejected item comments after the top 32 are announced and yes, I promise to not get offended that Sean wrote, "An item this crappy should at least promise to kill Hitler one or more times." Given time and effort demands on the judges, though, I think the current setup in which they're not released but people who are really curious can request feedback is probably the best compromise.
Here's why that won't work: even if you check the box, what you dont realize is that I may make comments about a similar item in my comments about your item. It would be nearly impossible (or even if possible, not worth the time) to edit out all those references. For instance, if you submit a necklace of deadly gloom and someone else submits a necklace of shadow's veil, it is likely that we might discuss that other item in passing in your item's thread. And there is no way to edit that out or have to go check to see if the other item also checked their box. Just too much work.
Plus, trust me when I tell you that you don't want the judges to feel that their confidential discussions will be public. You dont want to know or see. You get tons of feedback in the feedback threads. If the judges knew comments were going to just bulk be public you wouldnt have the same free discussion.
The way it works now is ideal.
The way it works now is ideal.
+1
I can say that having been on both sides of this contest now. Even as much as I might have wanted to see every last word the judges posted while discussing my item submission, it's far better that I didn't. Instead, I got to learn from the feedback threads, the bad item stereotypes (which became Sean's auto-reject threads), etc. In that fashion, I learned all those things on my own. If I'd read the judges' actual feedback, I might have taken too hard of a blow to the ego and psyche to make another go of it. Obviously, I'm much happier with how things worked out.
Trust me when I say you will be, too.
My two cents,
--Neil
RPG Superstar 2008 Top 32, 2011 Top 16
,
Star Voter Season 6, Star Voter Season 7, Star Voter Season 8, Star Voter Season 9
aka JoelF847
|
The only issue with that, I can think of, is that some judges may not be comfortable with their comments going public.Considering that some of my comments include, "this item's name is crap," yes.... ;)
Hopefully, for my item, the quote starts "This is a cool item, but..." before going on to say "this item's name is crap"
Crap... relates to bodily functions, vomiting, etc... Auto-Reject #19. Rather linear, to be sure.
No surprises, here.
The way it works now is ideal.
I did say that further in the post! :)
I'd like what I'd like, but I also understand the reasons why it's not workable in practice.
|
some judges may not be comfortable with their comments going public.
Considering that some of my comments include, "this item's name is crap," yes.... ;)
Just because I submitted the Crap Helmet, doesn't mean you have to tell everyone!! (joke of course).
Gosh....I hope noone really did submit a Crap Helmet :O
NOOOOOooooooo!
You said no items should include vomit; you said nothing about turds!
I am so hosed!
Is there no love to be spared for my Narcoleptic Bar-Stool?
(joke, honest!)
|
Dedicated Voter Season 6
|
I agree. And, there is even a chance they wouldn't give that honest of a comment. They might just say..."reject"!Some of the judges' comments go exactly like that.
Not mine, mind you. I have this tendency to spend 5,000 words per post. But, even on a few of them, some other judge got there before me. I'd read their assessment and just say, "Agreed. Vote to Reject." And then push the Reject button for us. In fact, I daresay I was the primary button-pusher this year. If you went to the Keep or Reject pile, odds are I pressed the shiny, red button that put you there. Okay. It's not really a shiny, red button...
looks like I got ninja'd by two people who were braver than me