Hey Judges, How is it going this first Round.

[Anthony Adam]

Hi Judges/Host,

As it's now Friday, I got to wondering the usual last week before reveal things, like...

Any tid-bits for us to voraciously devour?

Are you having a hard time agreeing on your choices?

Have you gone beyond top 64?

Come on, tease us a little :D

[Pamela Elizabeth]

Come on guys! Throw us a bone!

[Papasteve08]

I was also similarly anxious (and bored from waiting) which is why I started the top list swap thread, or as someone has awesomely dubbed it, the "Exit Poll" thread.

[mad_mac_hl]

Something along the lines of what Clark Peterson did last year would be great - the commentary he provided was very interesting I thought.

[Sean K Reynolds]

I've encouraged the judges to post their opinions here. Meanwhile, I'm doing some data analysis and will be posting some interesting tidbits in a few minutes. :)

[Russ Morrissey]

I've done 70 of the top 100; I'll finish tomorrow.

My own judging was less about style guides and more about what grabbed my imagination. So I'd personally be more likely to upvote something interesting but mechanically flawed than something perfectly balanced but which wasn't particularly original. The other two judges are better qualified to vote on mechanics and the like, so I went for ideas.

Things that tended to put me off (but not exclusively, so if you did this don't panic!) - Nounverb names, various cloaks which turn into swarms/mists/fogs/smoke, stuff I had to re-read a half dozen times just to parse, and stuff I found boring.

Things which grabbed me were things I wouldn't have thought of myself; stuff that fired my imagination; or stuff I found evocative of what they were trying to do. At least one I couldn't decide whether it was genius or bat-**** crazy, but HAD to vote for it for that exact reason.

We seem to have consensus about 3/4 of the time (just an impression - I haven't counted).

It takes a while - I spent several hours on it last night. Tomorrow I'll spend a couple of hours on my final 30.

[themantheycallcris]

Thanks for the comments Russell!

[Thomas LeBlanc]

Sean K Reynolds wrote:

I've encouraged the judges to post their opinions here. Meanwhile, I'm doing some data analysis and will be posting some interesting tidbits in a few minutes. :)

sean, if you would be so kind, I would like to see the number of voters last year in comparison to this year, number of each voter tag for this year, and an average of voters per day for both years. Please and thank ye!

It has also been more than a few minutes...

[Chris Shaeffer]

So demanding! ;)

[Taylor Hubler]

Looking forward to following this discussion.

[Patrick Renie]

Thoughts

There are a lot of "shadow"-themed items this year.

I've only seen one new figurine of wondrous power so far.

I vote with a sharp eye for proper use of rules language (i.e., mirroring the language of preexisting items).

Simpler items are tending to fare better. Items with multiple abilities are tending to do worse.

When I'm on the fence about an item, I take another look at its name. Sometimes a great/awful name can make/break an item.

Your sweat fuels my judgment engine.

[Sean K Reynolds]

Okay, the temporary power outage at Paizo today interrupted my statistics post, and it took a while to get the site up again, so I apologize for the delay. Here's the info I was going to post (which is about 5 hours out of date at this point):

• The judges have a consensus on 13 items (meaning for those items, all three judges have voted either "keep" or "weak keep").

• Within that group of 13 items are items the public voted #1 and #55.

• Five of the public's 10 highest-voted items have prominent spots in the (preliminary, informal) judge rankings.

[Judy Bauer]

Phew! Finally finished going through all 100. Man, you all love your shadow items...

Since most of the items did more or less ok in terms of grammar and punctuation (thank you!), I looked hardest at clarity of wording and explanation, and whether items' abilities made sense together and matched their flavor. Simple and well-defined beats out a random grab bag of useful abilities or amazing but hand-wavy item for me any day! Only a few items directly referenced our campaign setting, some more successfully than others. But I did take note when an item would have a clear appeal to members of a particular organization.

I look forward to seeing how the ranking plays out over the course of this weekend! (I think my favorites are safe—better check!)

Edit: Almost forgot! Folks—for your own sakes, in future contests, please spell your item's name correctly!

[Shadders]

Thanks Sean & judges. It's great to hear your thoughts!

[Tripp Elliott]

Judy Bauer wrote:

Edit: Almost forgot! Folks—for your own sakes, in future contests, please spell your item's name correctly!

So that seems like good advice

[James Risner]

Eagerly waiting on the public announcement of top 32!

[Zi'on Darkbane]

Sean K Reynolds wrote:

Okay, the temporary power outage at Paizo today interrupted my statistics post, and it took a while to get the site up again, so I apologize for the delay. Here's the info I was going to post (which is about 5 hours out of date at this point):

• The judges have a consensus on 13 items (meaning for those items, all three judges have voted either "keep" or "weak keep").

• Within that group of 13 items are items the public voted #1 and #55.

• Five of the public's 10 highest-voted items have prominent spots in the (preliminary, informal) judge rankings.

I was wondering why I couldn't get into the site today - and just after your little teaser out there to boot...

And yet you call it a coincidence...? Makes me start to wonder... :-]

Seriously, thanks for the info...

How about some more??? (Give them an inch...)

[Sam Harris]

Thanks for the responses, and for all your work, judges! Looking forward to the big reveal on Tuesday.

[Samuel Stone]

Indeed! Hopefully the power outage hasn't made things too much more difficult on the judges.

[Kalervo Oikarinen]

Why isn't it Tuesday already? The suspense is starting to get to me. :)

My two monster round candidates are nearly finished and no longer prove enough of a distraction.

[Curaigh]

Thanks Sean,
Thanks especially to Judy, Russel, and Patrick!

Consensus on 13? Lucky!

Is the break off point you're looking at the same as last year? More up votes than down votes if I remember correctly. If so I am glad that the submissions have improved from 89 to 100. :) (Congratulations #55)

So close now, this is going to be a long weekend.

[Oceanshieldwolf]

As I posted earlier, but the post was eaten by the outage:

Can we get a "?" put on the end of the thread title?

Anthony Adam, your alter ego would soundly slay you! ;)

[Anthony Adam]

Lol, i hate typing on virtual keyboards on a touch surface!

Thanks to all the judges for their feedback, and wow, top 100! Looks like we didn't make it easy for you ;)

[Scorba]

I am really curious as to what is item #55?

This a blind congratulations to #55.

[Russ Morrissey]

So I still have my final 30 to do (yesterday's website outage kiboshed my plans). I'm hoping to get them done today/tomorrow AM (my time).

[Oceanshieldwolf]

So of course I immediately made a thread asking "what's up with the site" without a question mark. Hubris or karma? And can you make an item out of it in less than 300 words? (formatted correctly of course).

[Papasteve08]

It seems we have lost some posts... SKR posted some interesting statistics last night on 1-100 ranks and their upvotes to not voted for differential...

[pH unbalanced]

Papasteve08 wrote:

It seems we have lost some posts... SKR posted some interesting statistics last night on 1-100 ranks and their upvotes to not voted for differential...

Here's what I remember from what he said:

The Top 100 averaged 1,000 votes.
The #1 item had a vote differential of +500 Votes
The #100 item had a vote differential of +200 votes.

[Papasteve08]

pH unbalanced wrote:

Papasteve08 wrote:

It seems we have lost some posts... SKR posted some interesting statistics last night on 1-100 ranks and their upvotes to not voted for differential...

Here's what I remember from what he said:

The Top 100 averaged 1,000 votes.
The #1 item had a vote differential of +500 Votes
The #100 item had a vote differential of +200 votes.

Thats how I remember it as well. Just a bummer it was lost! (I lost a few PMs as well, technology is a cruel mistress sometimes...)

[Sean K Reynolds]

Argh, website downage ate my followup post. Let me summarize:

Curaigh wrote:

Is the break off point you're looking at the same as last year? More up votes than down votes if I remember correctly.

Nope! The voter #100 item is +200 (upvotes minus downvotes). By comparison, the voter #1 item is +500. If the list cut off at 50%/50% we'd have a LOT more items.

FYI, each of the items in the voter #1–#100 was voted on about 1,000 times (some were as low as 930 or so, some got close to 1,100, but overall it's about 1,000 for each of the voter's best hundred items).

Good memory, pH unbalanced and Papasteve08. :)

[Kenton Abel]

Sean K Reynolds wrote:

The voter #100 item is +200 (upvotes minus downvotes). By comparison, the voter #1 item is +500. If the list cut off at 50%/50% we'd have a LOT more items.

FYI, each of the items in the voter #1–#100 was voted on about 1,000 times (some were as low as 930 or so, some got close to 1,100, but overall it's about 1,000 for each of the voter's best hundred items).

Good memory, pH unbalanced and Papasteve08. :)

Not sure if it is possible to go back in time to the statistics just before the cull, I would be curious to know how many of the final top 100 items were also in the top 100 before the cull.

[James Risner]

Kenton Abel wrote:

final top 100 items were also in the top 100 before the cull.

I'd say none, because I'd assume the culled were all more down than up

[Joel Flank]

Did all of the items post-cull get roughly a 1000 votes, or only those in the top 100?

[R Pickard]

Judy Bauer wrote:

Only a few items directly referenced our campaign setting, some more successfully than others. But I did take note when an item would have a clear appeal to members of a particular organization.

I'm curious... I recall a few years ago seeing advice that in the first round either to not reference Golarion or to not feel one had to, as not all judges were from Paizo and might not get the reference.

Obviously, the later rounds get very Golarion specific, for obvious reasons, but I'm just wondering if there's a shift even in the first round and items that reference Golarion are most desired, or it doesn't matter as long as the item is understandable and well-designed.

[Judy Bauer]

I wouldn't call it a shift. It certainly wasn't a free pass for me if it was a throwaway reference or the item wasn't otherwise strong, and as the earlier post notes, it's risky given external judges who might not know why something's a really good fit for the setting. But a well-designed item that also picks up some aspect of the setting and runs with it (even if not by name)—that was extra appealing to me personally.

But at minimum, items really have to be able to stand on their own and catch the interest of someone who doesn't know the setting. The reference should also be well-integrated into the item's flavor and abilities, and not too narrow or limiting. An item found in only one minor River Kingdom is going to be downvoted just like, say, an item that works only for sorcerers from a particular bloodline.

[R Pickard]

Cool, thanks for the clarification!

[Anthony Adam]

31 hrs 14 minutes to go :P

[frank gori]

Anthony Adam wrote:

31 hrs 14 minutes to go :P

I wanna be sedated

[Jeff Lee]

GM_Solspiral wrote:

Anthony Adam wrote:

31 hrs 14 minutes to go :P

I wanna be sedated

Well, I don't wanna be buried in a pet semetary, but we all can't get what we want all the time, can we?

[Landon Cole]

We'll have to calm down in a garden of serenity.

[Anthony Adam]

Joel Flank wrote:

Did all of the items post-cull get roughly a 1000 votes, or only those in the top 100?

I "think" the voting pairs is supposed to act like dealing from a deck of cards. All the items go into a deck. Two cards are removed and dealt to each pair request. When the deck is empty or has just 1 card in it, then the deck is refreshed to full and a new pair is dealt.

So in theory, the disparity between 1100 votes and 930 votes is attributable to the "no vote" button. I.e. the vote count is not "number of times it was paired", it's the number of times it was voted on either up or down.

If the distribution was purely random, then with bell shaped probabilities, the items submitted first and last could be seen a lot less then those submitted in the middle.

This is why I believe it's a "pool/deck" system.

[MrCab]

Anthony Adam wrote:

Joel Flank wrote:

Did all of the items post-cull get roughly a 1000 votes, or only those in the top 100?

I "think" the voting pairs is supposed to act like dealing from a deck of cards. All the items go into a deck. Two cards are removed and dealt to each pair request. When the deck is empty or has just 1 card in it, then the deck is refreshed to full and a new pair is dealt.

So in theory, the disparity between 1100 votes and 930 votes is attributable to the "no vote" button. I.e. the vote count is not "number of times it was paired", it's the number of times it was voted on either up or down.

If the distribution was purely random, then with bell shaped probabilities, the items submitted first and last could be seen a lot less then those submitted in the middle.

This is why I believe it's a "pool/deck" system.

I'm not sure about that. There are plenty of times I got the same item back to back. I'm pretty sure I once got the same 3 items across 3 votes, just with different pairings. There's also one item I definitely only saw twice overall out of umpteen votes, and I saw lots of spell in a can / feat in a can items (not to mention outright "breaks game balance" items) well over 5 times each across all votes.

That all said, I would love to know what the algorithm was. Maybe it was the deck, but across all voters instead of just one deck per voter, but that would make some of my pairings extremely unlikely.

[Wicht]

MrCab wrote:

Anthony Adam wrote:

Joel Flank wrote:

Did all of the items post-cull get roughly a 1000 votes, or only those in the top 100?

I "think" the voting pairs is supposed to act like dealing from a deck of cards. All the items go into a deck. Two cards are removed and dealt to each pair request. When the deck is empty or has just 1 card in it, then the deck is refreshed to full and a new pair is dealt.

So in theory, the disparity between 1100 votes and 930 votes is attributable to the "no vote" button. I.e. the vote count is not "number of times it was paired", it's the number of times it was voted on either up or down.

If the distribution was purely random, then with bell shaped probabilities, the items submitted first and last could be seen a lot less then those submitted in the middle.

This is why I believe it's a "pool/deck" system.

I'm not sure about that. There are plenty of times I got the same item back to back. I'm pretty sure I once got the same 3 items across 3 votes, just with different pairings. There's also one item I definitely only saw twice overall out of umpteen votes, and I saw lots of spell in a can / feat in a can items (not to mention outright "breaks game balance" items) well over 5 times each across all votes.

That all said, I would love to know what the algorithm was. Maybe it was the deck, but across all voters instead of just one deck per voter, but that would make some of my pairings extremely unlikely.

Of course, you don't know how many people were voting at the exact same time as you so as to make the deck shuffle through between your picks, so to speak.

[MrCab]

Wicht wrote:

MrCab wrote:

Anthony Adam wrote:

Joel Flank wrote:

Did all of the items post-cull get roughly a 1000 votes, or only those in the top 100?

I "think" the voting pairs is supposed to act like dealing from a deck of cards. All the items go into a deck. Two cards are removed and dealt to each pair request. When the deck is empty or has just 1 card in it, then the deck is refreshed to full and a new pair is dealt.

So in theory, the disparity between 1100 votes and 930 votes is attributable to the "no vote" button. I.e. the vote count is not "number of times it was paired", it's the number of times it was voted on either up or down.

If the distribution was purely random, then with bell shaped probabilities, the items submitted first and last could be seen a lot less then those submitted in the middle.

This is why I believe it's a "pool/deck" system.

I'm not sure about that. There are plenty of times I got the same item back to back. I'm pretty sure I once got the same 3 items across 3 votes, just with different pairings. There's also one item I definitely only saw twice overall out of umpteen votes, and I saw lots of spell in a can / feat in a can items (not to mention outright "breaks game balance" items) well over 5 times each across all votes.

That all said, I would love to know what the algorithm was. Maybe it was the deck, but across all voters instead of just one deck per voter, but that would make some of my pairings extremely unlikely.

Of course, you don't know how many people were voting at the exact same time as you so as to make the deck shuffle through between your picks, so to speak.

Very true. I just find it highly unlikely I would get the same items as often as I did.

21 hours 21 hour 21 hours 21 hours....

[Sean K Reynolds]

So... close... to being... finished...

(The decision rests on whether #43 or #45 is selected as an alternate...)

[Kiel Howell]

Oh you delightfully evil tease!

[Taylor Hubler]

#43... No... 45.

Wait. No... definitely 43. It is shinier.

Dang it... #45. Final answer.

[Voice Of Reason]

I think you should compromise and just go with #44...right in the middle.

[R D Ramsey]

43 is prime. That should settle it.

[Anthony Adam]

Be radical - choose both and just have an extra alternate this year!