I am curious about the "golden ticket" element of the judging. Is it really necessary? Doesn't the normal process of evaluation give us the strongest candidates? Several judges critique an item, they agree or disagree on the relative strengths and weaknesses of the item and consensus rules. The ticket seems to circumvent this process, allowing a judge to pick one item, regardless of what the other judges say, and put it through. Am I wrong in my assumption of how the ticket works?
I don't think the Golden Ticket mechanism leads to items that are a single judge's favorite leapfrogging over items that a lot of judges like. Rather, a lot of slots are filled with items that acheive wide enough consensus, the ones that distinctly stand out overall above the rest of the Keep pile. Then you've got a few slots left; by definition you've only got items left that didn't stand out for more than a judge or two. How do you choose between them?
You could, hypothetically, choose items that most judges "didn't dislike much" - basically saying each item has a "total score," and the ones with the highest score get in. But the Golden Ticket method does something different. It says, well, some items aren't well received by most of the judges, but one of them sees a potential spark worth nurturing. Let's give that spark a chance.
In other words, the choice here is between judges' pet favorites and generally-accepted mediocrities. The statement here, IMHO, is that the former have a much better chance of making for an interesting competition.
I dont think the golden tickets stop the strongest entries - they are used when the judges have the last few spaces to fill and they have a strong feeling about the designer behind a particular entry.
This means they only happen at the very end of judging, the strongest entries always make the cut.
Here is how we sort the keep folder:
An item can get in the keep folder with less than all 4 judges thinking it is great. Once there, they are ranked. At the time of this internal ranking, we revisit our keep/reject decision based on all of the items that were kept. All items that, at the end of that process, have all 4 judges support are automatic locks. Then all items that have 3 judges support are locks. Then we see where we are at.
This year, that got us 20 items with 4 or 3 judges agreement. That left 12 items for us to select from in the keep folder to fill out the top 32. Plus we wanted 4 alternates. Since there are 4 judges and 16 spots, each of us got 4 golden tickets. We went in order Sean, me, Ryan, Neil, using our golden tickets.
So the items that by consensus were the best were locked in the top 32 before we used golden tickets. We only golden ticketed from similarly situated items--items that had only 2 judge support. I kept an item in the keep folder that, in the end, I was the only one who liked it. I could have golden ticketed it, but decided not to since that would not be fair to the items that had support from 2 judges (even if not from me). So we used the golden ticket method to select from among the "2 judge vote" items.
Why golden tickets? Because we are all individual judges. We are not just tallying points. What appeals to me might not appeal to Ryan. What Sean loves, Neil might not. We each bring something to the table and see things just a bit differently and the golden ticket process is the time for us to champion an item from the keep folder. And we each had very different selections. That is part of the value of the diversity of the judges. Neil loves the craft, as you would expect from a freelancer; Ryan is more systemic, as you would expect; Sean thinks like a developer, which is his role; I am the idea guy, I like the fun of the game and the big concept. Those views reflect themselves in our ticket choices. But remember, we are ticketing from a pool of items that werent good enough to be automatic locks and yet still were keepers.
We have NEVER had 32 items that all 4 (or 3) judges agreed were the best, so the golden ticket process has been in place since invented by Erik Mona (well, not invented, how about "utilized") in the very first year of the contest.
By the way, NONE of the items in the Keep folder are mediocre. They are all good items. They just didn't have judge consensus. The golden ticket method is the fairest and most fun way to choose from among the remaining items. So even items that didnt get tickets are good items. Its just that each of the judges liked the items they ticketed better.
This is a lengthy process and the subject of much discussion by the judges and substantial review of prior opinions about the items.
This answer is from my wife!
I apologise for wifely bluntness, she doesn't mince her words, gawd bless her.
"Now look, doofus. Those poor hard beset judges work ruddy hard for no pay. Just look at their bald spots where you have made them pull their hair out.
The golden ticket they each get. Isn't it obvious? That's their payment. That's their one ray of joy where they can pick "their" favourite and ensure it's in. Surely they deserve one brief moment of unrestrained glee.
Imagine if you will...
Clark: I'm using my ticket on this...
Neil: Ack!
Ryan: You didn't?
Sean: He did, so I'm ticketing this one.
Neil: Sigh.
the judges are allowed those moments, it's what they have waited all month for.
And..
AND!
And, if they don't need to spend it, well, it's gold. Gold is good. Gold makes their ladies smile..."
At which point I stopped her before the hinting got more obvious...
If only it worked that way.
With all that creativity, tell your wife SHE should enter next year too. :)
...
This year, that got us 20 items with 4 or 3 judges agreement. That left 12 items for us to select from in the keep folder to fill out the top 32. Plus we wanted 4 alternates. Since there are 4 judges and 16 spots, each of us got 4 golden tickets.
...We have NEVER had 32 items that all 4 (or 3) judges agreed were the best, so the golden ticket process has been in place since invented by Erik Mona (well, not invented, how about "utilized") in the very first year of the contest.
Thanks (as always :) Clark.
20 seems low to me. Only about 60% of the top 32? Especially with the bar getting higher every year. I would not expect 32 lockins, and some years are probably better than others, but is this number about normal?
I agree that none of the items are mediocre. Even items I did not like came from people who frequently got my votes in later rounds. :)
The number is a little different, given that we've got 4 judges rather than just 3 like the first few years of the contest. So, getting all 4 judges to Keep something is lot harder than just getting 3 judges to Keep something like in the past.
That said, all-consensus Keeps are a fairly rare thing. It's never all 32 items. That's for sure. And, honestly, from a judging perspective, that's actually a good thing. Your items are being reviewed from multiple angles. Those who get high marks from judges who assess from their differing points of view are clearly Superstar-worthy.
For comparison's sake, the judges settled on early consensus locks with the initial Keep tallies last year. I forget how many that was...but it was less than 32, obviously. Then, we took a look at the next layer of items and revoted on the ones we wanted to see among the Top 32. Those established a new layer of "consensus," allowing us to round out the first 31 spots in the Top 32. That left just a single "golden ticket" for most of us to pick that last spot and an alternate. So, the process varies depending on what we're looking at in the Keep folder and how the judges want to approach it.
This year, we had more "golden tickets" to spend because there was less consensus among the remaining items after we identified the early picks.
Now there's a job I'd like... Golden Ticket Auctioneer.
So judges, what am I bid for 5 golden tickets?
All joking aside, I'm as Jealous (yeah, capital J) of you Judges and the fun you must have as I am of each years Top 32.
All joking aside, I'm as Jealous (yeah, capital J) of you Judges and the fun you must have as I am of each years Top 32.
Its funny...
The work is horrendous. It is a TON of time. And all of us are super-busy. It is really really really work intensive to review these submissions.
But we do have a ton of fun. The fun, though, comes from our interaction and from the passion we have for Paizo and for this contest. I love it. Neil loves it. Ryan and Sean, too. Our mutual enthusiasm for the contest really keeps us going. Plus, working with three other judges as awesome as Sean, Ryan and Neil doesn't hurt :)