| 6 people marked this as a favorite. |
Well, I'm almost done with my sorting. The judges should have our sorting done individually tonight then tomorrow we will use our rankings to select the top 32.
I just want to say, I think the community did a great job of sorting the good from the bad. Your votes clearly track, in my view, which ones are good and which ones are not.
Without getting in to the math and all that (since I dont know what Paizo intends to release and that is not my call), there are about 90 items that you folks sorted into the Keep pile, so to speak, being items that clearly had more wins than losses.
I have reviewed the items outside that pile (from 90-200) and I didnt find even one in there I would champion for top 32. There are some good items, yes, but none that make me want to bump out one of the higher ones.
So as a result, here is what I am doing. I am limiting my selection to the top vote getters (the top 90). I cant speak for Sean or Wolfgang but my guess is they feel the same.
So that means this year we are REALLY respecting the community input on our selection. Not only did you do a great job sorting items, but I (and likely we) am going to respect your rankings too and limit my choices from within that top 90.
Now, that said, I am not going to confine myself to the specific order based on votes. In fact, there are some items that we have all passed on for various reasons that are in that top 90 of items.
In the end, while this process doesnt really generate a good top 32 just by community votes, it sure narrows the pool down to a very good 90 or so items and I am going to respect that and I believe my fellow judges will too.
Just wanting to keep you all posted on how this new process works!
Great job everyone and I can't wait for you all to see the final top 32 items.
I will also say this, among the top 90 items, there are more really good items than I have seen in a while. Usually you get 15-20 keeps and the rest are good but not sure if they are ready yet kinds of items and as a result we have an easy top 15 or so then it is our personal preference on the balance. This year my guess is we will have nearly all the top 32 by consensus.
For instance, the top 3 vote getters based on your votes were weak keeps at best. I didnt get a strong keep until number 7 and 8, then number 9 was a NO from all of us.
What that means is this voting process was, in my view, a HUGE success in the community sorting items into what could be considered an initial Keep pile, but it still takes judges to rank within that pile. I have to admit I am very pleased with how the community input worked this year.
And the three judges are seeing these items pretty similarly. There is not a wide divergence of opinion by this year's judge crew.
As an impartial observer to the process (i.e., I was granted access to the judging chambers just to see how things were going to proceed this year with the new process), I too like how things seem to be shaking out. It's very cool to watch Clark, Wolfgang, and Sean work through this new approach. With 90 or so items in the Keep pile, that's reasonably close to what the judges established in prior years. But, as Clark indicated, with the quality going up again, it's cool to see more "locks" for the Top 32 this go-around.
Well done, voters!
Are the voting rankings going to be released?
I doubt it. But that is Paizo's call, not mine.
Thank you very much Clark. This behind the scenes info is really cool and it is very heartening to hear we did a good job sorting. It is doubly cool that you, Sean, and Wolfgang like our rankings! I hope we get to vote again next time.
As another example, an item that was 37 based on votes is a strong keep from all three of us and will be likely a top 10 lock.
| 3 people marked this as a favorite. |
Starts praying with everyone else (not 9, be 37, not 9, be 37)...
Seriously though, it is very heartening to hear this, I look to my list of items and am starting to feel confident that I may actually have a good hit ratio against the final top 32 :)
Thanks so much, Clark, for taking the time to give us a peek behind the curtain, so to speak. :) I agree with what theheadkase and Anthony said, it's really heartening to hear that the community did a pretty fine job sorting, overall.
I kind of figured this sort of approach was going to be best for determining an initial "keep" pile, and then the serious sorting could begin from there. I'd assume this allows the judges to spend more time deliberating the truly awesome items with fresh eyes. It's also great to hear that there are more easy picks.
And, though I know Paizo will never release which items got passed over, I can't help but be curious which ones floated to the top that still got an instant-veto from the judges. (Please not me, please not me... :P)
And way down at 59 I found an item I thought was a strong keep.
I'll also comment that while the top 90 ranking isnt controlling our rankings, it is generally a good guide. For instance, from about 1-30 there were a couple nos and a couple weak keeps, from 30-60 there were way more nos and fewer keeps or weak keeps, and from 60-90 there were very few keeps and almost all nos. So your rankings were a good general guide.
Marathon Voter Season 6
|
| 4 people marked this as a favorite. |
...And, though I know Paizo will never release which items got passed over, I can't help but be curious which ones floated to the top that still got an instant-veto from the judges. (Please not me, please not me... :P)
If it was my item, I'd obviously be disappointed but at the same time, I'd love to read the feedback why it got rejected by the judges. I think it would be a fantastic opportunity for learning; even though "my" item was a success in the public voting, "this" is why the judges looking at it with an experienced eye rejected it. That would be a very valuable design lesson for the public. If the Critique My Item sub-forum gets up and running, hopefully the judges might pop their heads into a thread or two for such feedback.
Best Regards
Herremann the Wise
I am glad the voting panned out well! Thank you Clark for the extra insight, it really eases the wait.
Actually, I remember thinking several times during the process that it was a good thing my votes were just one voice among many. There were quite a few pairings which seemed a simple matter of taste to me. Good luck to everyone!
|
RPG Superstar 2013 Top 16
,
Dedicated Voter Season 6, Dedicated Voter Season 7, Star Voter Season 8, Star Voter Season 9
aka DLandonCole
|
This is, I'm guessing, going to make it a lot easier for the judges - the further down the list, the less likely an item is to be superstar - but I really enjoyed being able to be involved at this stage and being able to see nigh-on all the round one entries. I saw a few entries that I really liked and I will be very interested to see if the community and the judges agree with me on those.
By the sounds of things, this system of voting will be around next year. I do hope so!
RPG Superstar 2012 Top 32
,
Champion Voter Season 6, Champion Voter Season 7, Champion Voter Season 8, Champion Voter Season 9
|
| 1 person marked this as a favorite. |
If the top 90 were to be posted, alphabetical order with no clue as to rankings would be to go. That way future contestants could see what the voters were liking. And I am curious to see if any camping, walky talkies, or make adventuring safe items made it up that high.
Thanks Clark for easing the wait!
Clark, would you like to see the voting continued in future contests? And percentage wise, how much less time did this years method take for you as a judge?
As another example, an item that was 37 based on votes is a strong keep from all three of us and will be likely a top 10 lock.
*dreams* If only it were me because #37 is my favorite number!
| 2 people marked this as a favorite. |
If the top 90 were to be posted, alphabetical order with no clue as to rankings would be to go. That way future contestants could see what the voters were liking. And I am curious to see if any camping, walky talkies, or make adventuring safe items made it up that high.
Thanks Clark for easing the wait!
Clark, would you like to see the voting continued in future contests? And percentage wise, how much less time did this years method take for you as a judge?
There were a couple "make adventuring safer or easier" items. But you know what, that is ok. Those are not absolute rules. In fact, there is one item that is a spell in a can but it is such an awesome spell in a can it is the exception that proves the rule and all three judges agreed it was great.
I wouldn't mind seeing the voting continue in future years. It was WAY less time for me, but I have to admit I always enjoyed the sheer torture of round 1. But then I'm kind of wacky that way.
|
RPG Superstar 2012 Top 32
,
Champion Voter Season 6, Champion Voter Season 7, Champion Voter Season 8, Champion Voter Season 9
|
And I am curious to see if any camping, walky talkies, or make adventuring safe items made it up that high.There were a couple "make adventuring safer or easier" items. But you know what, that is ok. Those are not absolute rules. In fact, there is one item that is a spell in a can but it is such an awesome spell in a can it is the exception that proves the rule and all three judges agreed it was great.
There were 2 camping, 1 walky talky, and 1 safe adventuring items that I really liked. Can't wait until Tuesday so I can see the SIAC that made it due to greatness.
I'm curious to know which item made the top 32 and see if I remember voting for them during the vote. At Marathon voter, I would be surprised if I didn't see at least one of the top 32.
RPG Superstar 2012 Top 16
,
Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Star Voter Season 9
aka Locke1520
|
I'm really happy to hear the voting worked to make the judges jobs easier and as Thomas said this does ease the wait until Tuesday's announcement.
Starts praying with everyone else (not 9, be 37, not 9, be 37)...
Yup...although I might be ok with 59 too ;D
|
RPG Superstar 2012 Top 16
,
Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Star Voter Season 9
aka Locke1520
|
I'm curious to know which item made the top 32 and see if I remember voting for them during the vote. At Marathon voter, I would be surprised if I didn't see at least one of the top 32.
I'm also kinda hoping to see some new ones I may have missed.
Maybe this been asked before and I missed it but do we have a number of actual entries (estimated or otherwise)?
| 3 people marked this as a favorite. |
What Clark said.
Except the part about liking the sheer torture of Round 1. :p
I think I'll do a post-mortem on the not-quite-Top-32 like I did last year. It'll be interesting to comment on the ones wedidn't choose and why we think the voters voted them up. In several cases, it was "yes, this is a useful item, so I can see why people voted it up, but it's not a *superstar* item." For example, a ring of three wishes is a useful item, but it's not superstar--there's nothing clever or innovative about it... but people might vote on it because it's so useful.
I'm also very happy to hear that the new system worked out so well. A top 90 for the judges to paw through is certainly better than the whole of entries (and now I know a little of what that is like).
I'm wondering if #90 is really a keeper or if it was just where they decided to cut it? And I wonder what the furthest number down the list that a member of the final 32 will be.
Thanks Clark,
Always eases the tension when the judges pop in with this kinda info :) The judges have been conspicously quiet & 'tis good to hear the rahrah.
I too would be interested to know more about what we totally missed :)
As another example, an item that was 37 based on votes is a strong keep from all three of us and will be likely a top 10 lock.*dreams* If only it were me because #37 is my favorite number!
D'oh! My lucky number is 9. I kept thinking I dinnae need to be ONE, 9 will suffice :)
I'd be curious to see some of the math breakdown, how items clustered and so on.
For example, item 90 had xxx more points than 91, or where ever the big breaks in votes occurred.
Purely curiosity, really.
| 1 person marked this as a favorite. |
It has been an interesting process this year. You are right, we judges have been pretty quiet in large part because we dont want to influence the voting in any way. But now that part is done so I can flap my gums. And, as one of my favorite succubus minions likes to tell me, I do like to hear myself talk. :)
| 1 person marked this as a favorite. |
D'oh! My lucky number is 9. I kept thinking I dinnae need to be ONE, 9 will suffice :)
Just so you know, your ranking just gave us a pile of items. Your ranking of those items isnt going to have any bearing on me. For instance, if I have two good items and I'm on the fence about which one I am going to promote, I am going to do it purely on the item and what I see with the item. I will NOT take into account that one was rank 45 while the other was rank 17. That wont matter to me at all.
And I just found a real serious keeper at 68. I think all three of us agree on it, too.
Any phrases driving us bonkers? No, we didnt look at as many of the bad ones as you guys did. :) But I did a lot of reading of ALL the items so I got a pretty good taste and I read with some enjoyment your thread on that topic. I can't say, I dont want to give anything away.
| 1 person marked this as a favorite. |
By the way, there are a bunch in the top 90 that are real close. I noted in a bunch of them that I said "no" to that I hope the author regroups and tries again next year. Really, I mean it. Either too specialized or seem like they would fit a particular adventure and not really done for RPG Superstar.
I think, at least my gut says, overall quality is up this year at the top. Normally I have seen the quality rise at the bottom. This year the increase, to me, is more notable at the top.
Contributor
,
Dedicated Voter Season 6, Dedicated Voter Season 7, Dedicated Voter Season 8, Star Voter Season 9
aka Steven T. Helt
|
Alright, so assuming my Mrph of Mrphing made it into the top 90 (fingers crossed), I can hope to be number 37, 59, or 68. That's a 1 in 30 shot!
Now, what I'd be interested in seeing is friends and families of the judges having a phrases driving them bonkers thread during RPGSS. :)
| 1 person marked this as a favorite. |
Actually, for me you want to be:
4, 5, 6, both 7 and 8 big time, 11, 13, 14, 16 (loved it even though when I read the name I thought I'd hate it), 22, 23, 25, 31, 34, 37 (loved it), 42, 43, 44, 45, 55, 59, 68 (surprisingly cool, or should I say hot :) ). If you were those numbers, you will likely be happy. At least from my standpoint. Those don't include my weak keeps that will likely be part of our discussion tomorrow.
That's 22 sure fire keepers from me. I'm not sure I've ever had that many before. And even the weak keeps I have are really, really good.
You have to remember our standard is sickly high. Like, crazy high. So even a "weak keep" really means "this is way better than your average item in a book, it is superstar quality but its not a lock for the top 32 though it should be considered for a slot."
Oh, and the item ranked 1 was a No from me and 2 and 3 were weak keeps.
Hmm, trying to remember if there were any heat-related items that really grabbed me. Nothing's coming to mind -- a couple cold ones, but no heat -- though I guess we wouldn't really be able to talk about it yet anyway, even if I could think of one.
Hmm, trying to remember if there were any heat-related items that really grabbed me. Nothing's coming to mind -- a couple cold ones, but no heat -- though I guess we wouldn't really be able to talk about it yet anyway, even if I could think of one.
There was a container and another item you wore with a few gray items.
And a strong keep from me at 85! Wow, finding a gem at the bottom of the pile. Now that was an item with some flaws but boy oh boy did it have mojo!!!
|
RPG Superstar 2012 Top 16
,
Marathon Voter Season 6, Marathon Voter Season 7, Marathon Voter Season 8, Star Voter Season 9
aka Locke1520
|
Hmm, trying to remember if there were any heat-related items that really grabbed me. Nothing's coming to mind -- a couple cold ones, but no heat -- though I guess we wouldn't really be able to talk about it yet anyway, even if I could think of one.
Ditto. but I'm probably going to smack my head, "Oh yeah! That one!"
Well, I'm done with all my comments and assigning of value. I put either "strong keep," "keep," "weak keep," or "no" on the top 90. Now Sean is making a table with our votes by item. We will, from that, build our list of consensus items. Then if there are spots open we will golden ticket items in. We will do that tomorrow.
All in all, the quality was up this year at the top end for sure. Maybe a couple less "oh my gosh wow" items but still a couple but then a really strong middle batch. Every year I think our top 10 are really good. I'm not sure our items 10-25 have ever been this strong.
Good work everyone! And big props to the voting community here. You guys did a great job judging.
Oh, and the item ranked 1 was a No from me and 2 and 3 were weak keeps.
I find it very interesting the #1 item was a "no" for you. Not interesting in that I do not like this; I actually find it invigorating. I think it qualifies the process in that your approach to these items is based on a significantly different standard than the masses (Me being one of those unwashed... Actually, I took a shower so we'll just keep the comment as figurative. Rabbit trail, anyone?).
One of the other unwashed (figuratively) made a thoughtful statement regarding viewpoint; being thankful that it wasn't just up to him. I agree. We unwashed (figuratively/literally) have very different leanings, but there are some items that will out. We can trust that the mass process followed by a review from the professionals has enhanced the process.
Additionally, you went down 200 deep to ensure the process worked. Thanks for doing this.
Alright, so assuming my Mrph of Mrphing made it into the top 90 (fingers crossed), I can hope to be number 37, 59, or 68. That's a 1 in 30 shot!
** spoiler omitted **
I'm pulling for you too Steven. :)
Well, I'm done with all my comments and assigning of value. I put either "strong keep," "keep," "weak keep," or "no" on the top 90. Now Sean is making a table with our votes by item. We will, from that, build our list of consensus items. Then if there are spots open we will golden ticket items in. We will do that tomorrow.
All in all, the quality was up this year at the top end for sure. Maybe a couple less "oh my gosh wow" items but still a couple but then a really strong middle batch. Every year I think our top 10 are really good. I'm not sure our items 10-25 have ever been this strong.
Good work everyone! And big props to the voting community here. You guys did a great job judging.
Is there a definite break betwen the 90 & 91st rankings? Or did you just pick that as a good stopping point (maybe as 3x the needed number)?
Either way I'm glad to hear the entire field is getting stronger :)
Either way I'm glad to hear the entire field is getting stronger :)
For some reason, I hear a song from Rocky I...umm... 2... or 3... Maybe it wasn't a Rocky movie.
RPG Superstar 2010 Top 16, Contributor
|
It'll be curious seeing the "Judge my Item" thread this year, it's bound to be a different flavor since the judges won't really be exposed to many of the items.
If the feedback is anything like in the past, many people are certain to discover if they made the first cut at that point.
Good to hear the judging went well. Cheapy was talking about statistics, what I'd love to know is how long it took for those top 90 to 'float' to the top.
It'll be curious ... what I'd love to know is how long it took for those top 90 to 'float' to the top.
RIGHT!
Is there a definite break betwen the 90 & 91st rankings? Or did you just pick that as a good stopping point (maybe as 3x the needed number)?
Actually, I rounded to 90. It is actually 89. After 89 there is a legitimate mathematical difference from there down. I dont want to say more, what Paizo decides to release is up to them. But I would say it was a clear line of demarcation based on the voting numbers. And the other judges agreed with my proposal.