How I judge

[MicMan]

I am describing my judging process to help you understand what goes on during the minute that you have my attention.

Hopefully this will allow you to compare this to your own and likely discover how different people can be.

I fully expect others to chime in sharing their own methods which would help setting priorities especially to novice submitters.

So without further delay:

Phase A: First Contact
There they are, two shiny new creations, but which to skim over first?

This is actually an important question because the first item I skim over is the one with a head start. If that item wins me somehow (more on this later), the second one must be better to regain momentum.

But what can you do to let me read your item first? Well, that much is easy. I read the titles of both and the title that seems more promising in an evocative and slightly mysterious way usually gets me. The exception is the very concise title that promises something thematically fitting, yet still interesting.

If none of this can be applied to both titles I skim over the shorter entry first.

Phase B: the skimming
Well, I have to admit I do NOT read new entries, I skim the text to find out the EFFECT as I am an effect junkie!

And now for the SHOCKING TRUTH:
The effects of almost all items can be described in one sentence. Period.

If you are using more than one sentence, think WHY. Do you use your precious words because you want to cover every single if and when? If yes, you risk cloaking an uninteresting effect in explanations, which is bad. Your effect must stand alone and be cool. Identify the absolutely necessary specifics and nothing else. I don't want to read that your boots only work when being worn.

The best effects are the ones that let me think "oh, I ALSO could use this to do that" even after I thought about the effect the third time your item came up.

Phase C: skim deep
OK, your item won me over once or twice against bad items on its effect alone but now its up against some serious competition. So I now look after not only what your items does but also how it does it!

And, yes, up to this time I don't care whether your item confuses your opponents with butterflies or with cucumbers. This comes now.

Be evocative and, above all, use fitting fluff. "Arrows of healing" that you must fire at your allies make for some nice very long range healing effect, but here the fluff simply does not fit, regardless of any wordy explanation you tack on.

Also don't be too obvious. Candles that shed light are fitting but are also boring. Some types of items are limited, such like scabbards, scroll tubes and quivers, and thus my attitude towards another "Scabbard of Storing" is usually very bad.

This also is the place to judge the balancing. This is a funny one. During the first contests the trend was to under balance items, many were very powerful and even game breaking. Now the opposite is true more often than not. Either the effect is underwhelming or it is overbalanced. Imho it is much better to shoot for the stars than to play it safe here. Playing it safe will 100% not get you anywhere, shooting for the stars, if done right, will.

Phase D: you really got me now
Congratulations, your item is good (imho). It has a crisp description of an exciting effect that offers something to the reader even on the second or third pass. It is balanced and the fluff fits and isn't overdone.

I will now, and only now, read it word by word. Only now will I pay attention to the pricing, aura, caster level and to other details. This is (imho) superstar level, the only question left is just how superstar it is.

Disclaimer
As already mentioned, this is how I judge. It is NOT how everyone should judge, it is not even the best way to judge, it is just the way that I like best.

[Anthony Adam]

Mine is similar this year - each step is repeated step by step for both items in the pair...

Warning - Wall of text alert, you have been warned ;):

1) Does your item grab me out of the box - does the name look good on my character sheet and will it sound wicked when say I am using it. A certain degree of template use is expected as well as paragraph breaks at power changes and the like. Basically is my first impression cool name, scan-able item for table use?

2) How MUCH do I WANT the item? If it wakens the munchkin in me as a player, then I begin detailed scrutiny of the crunch of the item. If I merely want it over similarly priced items without gagging for it, then you have struck the right balance in my eyes.

3) Is it visual, cinematic, evocative? Does it make me cackle with demonic glee? At this point you have good name, tight game balance and visuals that pull me in.

4) Asses the core theme - is it just repetition of a spell of feat or is there some subtle tweak to the rules, some surprising twist or turn I wasn't expecting?

5) Look at the price - does it look correct for where it fits in the wondrous item table (increments of 10, 50, 100, 500, 1000 etc. If your price looks like 13,476gp chances are you applied the formula but then didn't assess where it lies in relation to other items around that price.

6) Assess the items integration into the game - is it generic enough to be wanted by multiple classes, does it scale well or targeting a sweet spot in a characters adventuring career? One of the traps I fell into last year was making a "single class" item - it cost me the votes of quite a few voters who don't like/play that class. So I try to look beyond this and determine if it could be tweaked to be more universally used.

OK, in the case of a tie, (very rare)

I turn to the technical aspects of the design, correct auras that match spell and caster level, not putting material items in the construction line at the end (they should be obvious from your opening description), consistent wording of construction elements with existing items.

Things in the body description, are the capitalized when they should be, italicized when they should be? Are feats sorted alphabetically, are spells sorted alphabetically, are multiple auras sorted alphabetically.

8) If I still haven't broken the vote deadlock, I turn finally to the template use. Is it obvious you read the rules and used the right template? I know Sean said not to worry about the new format, but the rules did specifically state, and provide, the required template. That said, I only use this as a tiebreaker if the items are still tied.

In my voting, I would estimate my vote between two items being cast as follows

Step 1 - 1 vote this year.
An item name was so bad it even made me, king of bad names, cringe and vote - 1 time this year - just once.
Step 2 - 3% approx of the time.
Some items were simply too gonzo/OP for me to vote for
Step 3 - 20% approx
Quite a few items were simply visually stunning
Step 4 - 60% approx
Once you have studied the core theme, the items tended to naturally separate, the vote was most often clear here.
Step 5 - 15% approx
Ahh, the mystic art of pricing. Some were obviously wrong, even to me
Step 6 - 1% approx
When a few items reached game balance for separating them, more often than not, they were equally balanced as each other.

Template deciding factor came into my votes <1% of the time. Yes, I was sorely tempted, but forced myself to look at every item in an equal way and not dismiss a single one.

This voting is hard to do, your gut and instincts want you to vote immediately on some pairings, but if you can suppress that urge, you often discover a few wonders you might otherwise have missed.

Oh yeah, this is why my voting is so slow too :P

[Andrew Marlowe]

How I've Been Voting

I "skim" over both entries: I look at length, title, first few lines and something will catch my eye in one of the entries. I read that entry first...Lately its when I come up with a new item but at the beginning it was more intangible and I tend to try and read the least impressive entry first.

My Priorities

First Impressions At this point I read the central text and ignore most of the game data in the formatted sections. I'm reading for the following questions: Does this entry grab me? Does it have a cool cinematic element or a novel mechanic? Does it solve a problem that doesn't need solving or muck around with action economy? Also does the entry have unnecessary back story? Is it really a wondrous item and not something else. Does it avoid the obvious traps (MIC, SAK, SIAC etc.)? What is it's ease of use?

Interest Having read the entry would I be excited to use this in an adventure and/or would I be excited to receive the item in a treasure horde. Does the item make sense? Not only thematically but would someone really make this thing? (Items with egregious built in flaws tend to fail this last question.)

General Format I'm not looking for nit picky details here but did the designer use the correct format and the provided BBC? Again I'm pretty forgiving at this stage I'm not looking for correct case and italics in the body of the entry just the presentation provided in the instructions...Unless someone really gets it right because that really stands out!

Both Items Are Cool (or Both Are Not Cool)I can usually make my decision in the first three steps but sometimes not at that point I need to dig deeper. Are the spells good choices for the effect? Does the cost seem reasonable?

Get Nit-picky Only in extremely rare cases are things so close that I need to really go over the items with a fine tooth comb.

A couple of final things to note

None of this is an exact science and I don't necessarily do things in this order every-time ... this is just how I tend to do it.

Sometimes a designer does everything right and the item just doesn't connect with me and I down vote it a lot and occasionally the revers is true.

I'm certain my voting habits yield some really irrational results on some of the close items. As if some of my likes lay out like overly complicated R-P-S.

[Ariax]

I read every word of each entry, but then I rarely re-read any of them on subsequent views. I just refer to my list and see where the two items are relative to each other. If they're pretty close together, I'll re-read them and reassess their positions on the list. But that's not often necessary.

This year I decided to score every item on a scale from 1 to 7. The factors used to determine scores are (roughly in order of importance):

Interesting idea?
Good rules implementation?
Would players want it?
Would GMs hate it?
Well written?
SIAC/SAK?
Silly/game-breaking?
Follows format?

There are a lot of interesting ideas that ended up with a score of 4 or lower because of flawed (sometimes deeply flawed) mechanics. There are a few items whose core idea was so awesome that I wanted to give them a 7, but then they stumbled on some detail of implementation or formatting.

I think my own entry is probably somewhere between a 5 and a 6. The mechanics are sound, but I overworked the idea and didn't do enough to convey the visuals I had in mind. Plus I could not, for the life of me, come up with an evocative name.

[Scarletrose]

I guess my way of judging goes like this.

1) Detect autorejection
This is both objective and personal. Objective when the object violates the rules (as in it is not a wondrous item).
Personal is the kind of items that have a quality that I think should be rejected.
For example, this year every time I read about an item that involves being dead before the effect triggers is an auto reject as is either a way to make the character avoid death, or an object that just let's you die but has some kind of retribution effect (something that I don't think many adventurers would pick as they usually pick items that help them stay alive and do things while they are alive).
If only one of the two object has something that make me autoreject I usually don't read the other in detail. if non of them is autorejected (or both are) I pass to phase 2.

2) Detect broken items
Basically, is one of the items potentially too powerful? I'm not even talking about potential artifacts. I'm talking about items that in the hands of the right player becomes an "I win" button.
Again, if one is broken I vote the one that is not. If none or both are I pass to phase 3

3) Detect items that potentially ruin GM plans
As the judges said. Objects that prevent you from getting lost, that prevents you from getting into trouble, in general an item that removes the need explore and go into an adventure.

4) Detect items with missing information
How many uses. how much range? using what kind of action?
Not every item needs all these data, but some do and is nowhere to be found.

5) finally look for the most unique item

Of course sometime I find an item that really get's me and I give it a pass on some of these phases but this is usually how I vote

[Garrett Guillotte]

Find the core concept. What's the the one-line description of what this item is? If I can't find it, that designer's dug themselves a hole.

Find the mechanical hook. I skim until I see what the item actually does. I'm looking for a designer who can do something clever and describe it clearly, regardless of the complexity.

Read it for flavor. How cool is it? I'm looking for a designer who can provide well-written visuals that compliment the mechanics.

I usually vote around this point. When both items are equally good, I look for tiebreakers.

Check the price. Once I know what it does, is the price rational? Do the cost and price line up? Does the item do enough, or too much, for its price? What would happen if I bought a six-pack of these? Could a whole party afford one, and if so, does everyone having one change my impression of the mechanics? I'm looking for a designer who applied their finished concept to actual gameplay.

Is this a manufacturable item? Wondrous items are like top-shelf booze--they're special, flavorful, strong, and a little (or a lot) expensive, but most good shops are going to have at least one. What would be necessary to manufacture it? What are the ramifications of having dozens, hundreds, or even thousands of this item in the world? I'm trying to see how far ahead the designer thought with this item.

If I'm this far, I'm starting to examine both designers for their viability in the rest of the contest.

Writing quality. I'm looking for the designer with the tightest writing, and I'm looking ahead to the monster round to determine which designer has the best shot of impressing me.

Bullet holes in the designer's foot. Does anything in one item put me off of it compared to the other item? A bad visual, a less-optimal mechanical decision, unnecessary fluff?

Mental exercises. Short of actually playtesting, I try to use the items in my head and see if anything obvious sticks out as an issue. I'm looking for the designer who didn't playtest their item.

I don't look for template or formatting issues until I get this far. I've found that designers who can't follow the template instructions usually fall over in the more subjective parts of the process anyway.

[Samuel Stone]

First off, I just glance at both items. If the formatting is obviously wrong, I tend to read the correctly formatted item first.

I usually give both items a super-quick read-through/skim in order to figure out what they do. The simpler it is for me to understand the gist of an item, the better. A good example is the [url=http://www.d20pfsrd.com/magic-items/wondrous-items/wondrous-items/m-p/mask-of-giants]mask of giants. This item is mechanically rather complex, but the core idea is simple: you can wild shape into a giant. If I needed to read through the item several different times just to understand "you turn into a giant," that's a pretty big negative towards the item.

Then I read fully through both items, and decide which idea is better based on several criteria:

-Flavor; is the item evocative, does it create a cool mental image, etc.
-Mechanics; does the item break the game? Is it overpowered? Underpowered?
-Use; when would I use this item? As a GM and a player, I should be able to get some use out of the item, even if it's very situational. Back to the giant mask, there's an obvious combat use, but it could potentially also be used by a druid trying to infiltrate a giant's base. If a creative player can potentially use the item in multiple different ways, so much the better. On the other hand, it could be fun to use the item for BBEGs. What if the big bad evil giant turned out to actually be that big bad evil druid the party encountered three sessions back? In other words, the best items, imo, create potential for creativity.

If the items are too close for me to decide, usually small aspects decide. Garrett outline this much better than I can, and my thought process is very close to what he posted under his "tiebreakers" section.

[Chris Shaeffer]

Judge ye not, lest ye be judged. ;)

Short Version: I pick what my gut says I like most (or dislike least), with the subtext of "Would I want to play an adventure written by this person?"

(Long version will have to be on my lunch break.)

[Kiel Howell]

I look at the names. Which name do I like better?

I read the entries, not a detailed scrutiny but a solid read. Which one is more interesting?

At this point I think, what was the inspiration and what is idea they are trying to get across. If I can nail down the inspiration (in my case last year, Gandalf smoking his pipe) and it is not evocatively written or is straight copied...pretty much an automatic downvote. Is the idea and design space creative? If I haven't seen it done or it is done in a way that is just flabbergastingly well done...automatic upvote.

Balance. Is it easily abusable and designed to "win" the game? Then I won't like it. Does it do nothing at all? Then I won't like it. Does it do something AND it has well balanced restrictions? Then I will like it.

Template scrutiny. This is pretty much just a tie breaker. Yes, I will notice on my initial read through if you didn't format, but I won't judge then on it. Unless you have a really lovely idea, this will be your tiebreaker (there is an item this year that has no formatting that I LOVE).

If I make it this far without a choice...gut feeling. I trust my instincts.

[Joseph Kellogg]

I typically choose by throwing darts. The major downside is that my screen is nearly unreadable now.

[Feros]

Anthony Adam wrote:

This voting is hard to do, your gut and instincts want you to vote immediately on some pairings, but if you can suppress that urge, you often discover a few wonders you might otherwise have missed.

Oh yeah, this is why my voting is so slow too :P

And why mine is so fast. I vote so many times, I can reread and revaluate everything multiple times over. There are many number of things that I couldn't stand on first perusal that grow on me, and vice versa.

Most stay the same, I will admit.

[frank gori]

typos > formatting errors > pricing issues > construction issues > well executed SIAC > abstract rules systems > safe choices that do not attempt superstar > overpowered > underpowered > flowery overwrought language > distracting grammar errors > obvious ripoff items > well executed joke item

combinations of errors can tip the scales

[Thomas LeBlanc]

1) Left Item
2) Right Item
3) Neither

Vote: 1d3 ⇒ 1

Another vote down!

[Eric Hindley]

I have a pretty brutal system, I'll admit. I read one item (pretty much at random), and form an opinion. I like it or I don't. Then I skim the other item, looking for keywords (effects, spell names, class features, etc.). If I find a reason to like the item (in the case I hated the first item), I vote for it. If I find a reason to dislike it (in the case I liked the first item), I downvote it. If they seem close, I read them again and reassess. Basically I give it what the Superstar judges did in the first few years- give me a reason to vote against you and I will.

This gets easier as I see more items, because I know them better.

[MicMan]

Eric Hindley wrote:

...give me a reason to vote against you and I will...

Yes, this is actually much easier than the other way round.

But I also found that flaws are forgiven much more if you are short and concise. If I need to skim over a lot of unimaginative rules text to find a good effect hampered by a slight flaw I am much more inclined to vote it down than having the good effect handed plus flaw to me on a platter.

[pH unbalanced]

I vote intuitively, but I can analyze my decision-making process after the fact.

The factors that sway me are:
Is it cool?
Is it broken?
Is it understandable?
Is the pricing out of line?
Will it cause too much work for the GM?

Only if the items are tied after evaluating on those factors do I start looking at:
Are there errors in the entry?
Are there errors in the formatting?

In addition, I have dealbreakers that I will always downvote:
Anything that physically disgusts me (creepy is good, icky is bad)
Anything that requires a user to perform an act I would not allow a villain to perform onstage in my game
Anything with alcohol

Those last are visceral reactions, they are not necessarily value judgements on the items or the creators.

EDIT: Oh, I also keep kind of a mental ledger of how many times I've voted for/against particular items. If it comes down to a tie, and I know that I have downvoted a particular items several times in tough pairings, I am much more likely to give it a 'makeup' up vote. And vice-versa.