This year's item cull?

[Thomas LeBlanc]

Did you like the big cull in this year's contest? Vote yes or no by liking the appropriate post.

[Thomas LeBlanc]

YES! I like this year's cull.

[Thomas LeBlanc]

NO! I didn't like this year's cull.

[frank gori]

Thomas LeBlanc wrote:

Did you like the big cull in this year's contest? Vote yes or no by liking the appropriate post.

The ones that are so offended by the cull that they left... aren't going to vote, because they left.

[Thomas LeBlanc]

Well that means more votes for liking the large cull and maybe a chance of it happening again...

[Jeff Lee]

The neutral response.

[frank gori]

It vastly improved the items we were voting on and I liked it so I voted it up. I like the idea of a weekly escalating cull even better (10% then 25%, then 50%.) Improving the process incrementally and sort of creating some drama toward the end.

I recognize however there are flaws with that scheme. There are also generally flaws with the deep cull as it seems to have scared some folk off entirely and it actually killed some really good items (like half of the best rings.) For this reason despite liking the Yes I will also like the No.

[Eric Morton]

GM_Solspiral wrote:

The ones that are so offended by the cull that they left... aren't going to vote, because they left.

Did that actually happen somewhere? I've seen a fair number of people disappointed about getting culled and a few people raising questions about the merits of a deep cull, but I don't recall anyone actually getting mad about it.

[Mikko Kallio]

There are still hundreds of items being voted on, so even if the good items hadn't been culled, they wouldn't have made the group of 100-ish items from which the top 32 is selected. The end result for those items is more or less the same. (unless the process is somehow different from the last two years.)

[Isaac Volynskiy]

Well, there have been just over 800 submissions (assuming the community generated list is close to accurate), and the forum has maybe...15 vocal members?

We probably won't know how the majority of people feel, just this sample.

[Chris Shaeffer]

Every year we are surprised. Without fail. :)

[Brigg]

This being my first year, the idea of a cull rang positively with me. Seeing that over 50% of the items were culled makes the idea of future contests involving culls a little more intimidating.

But that shouldn't stop anyone.

[Oceanshieldwolf]

I would vote

YES I was in favor of "A" cull. Has worked the previous two years to cut away the items that have performed poorly with the voters such that no further voting will redeem them.

NO I did not like when the cull happened. Personally I was burnt out and pretty much reached Marathon around the time the cull happened. If it had happened earlier I would have notched up waaaaaaay more post-cull votes cast, even though I'm pretty sure my item would have been caught in any cull, regardless of when it occurrred.

MAYBE it could be done differently next year. (Actually it's a definite - it is completely possible and probable that the cull will be carried out differently next year. :p)

[Mikko Kallio]

An escalating cull as GM_Solspiral suggested would probably seem less intimidating to new contestants, and if the first cull happened earlier on in the voting, it would probably keep the voters more motivated.

EDIT: Though of course I have no idea how much manual work a cull involves - if it's mostly automated, I support the idea of having smaller culls but more culls.

[Brother Fen]

Being culled is part of being a freelancer. Your ideas can get rejected at any time for reasons you may or may not agree with. People that want to work in any creative field need to accept that.

What might help people is perhaps a Junior RPG Superstar In-Forum practice competition where there would be open voting in a thread where everyone posted their items and then offered feedback. It could perhaps help people learn to accept criticism and rejection.

This is something we as a community could implement at any time.

[Nickolas Floyd]

I've been a fairly vocal opponent of the deep cull, but to be honest, I don't know if was more of a good thing or a bad thing. My gut says that it disenfranchised many voters who knew that they would not see there own items anymore.

Did it make those who were not culled even more enthusiastic about voting? Maybe. Does this offset the disenfranchised voters? Maybe. Is it better to have more people voting on more items or fewer, more fervent voters, voting on fewer items? My gut says it is better to have more of both. If not for total number of votes, for keeping more people interested in the competition for this year and the next.

Just about everyone had favorite items culled. Even if a majority of the items culled do not have a chance to make the top 100 to be viewed by the judges, what is the harm in giving more contestants hope for longer? Even a small cull will bring excitement to the voters who will still see less poorly designed items. I'd rather there were more voters excited, than fewer.

There are also items in the competition that click with certain voters the third or fourth time they are viewed. I've seen posters mention this several times. Maybe these initial votes kept them down for a while, but they will grow as more voters get it. Do these items have a chance of making the top 100 by the end of the voting? I don't know, but they should given the chance. These are golden ticket kind of items, that might take the right judge to champion or the right number of reads to really make an impact on someone.

Not knowing what the bigger cull did to the voting statistics I can't say for sure what impact it had there, but it is my opinion that culls should remain small to keep more people involved and remove only the worst items. I would definitely support two smaller culls, but I definitely think culls should not eliminate more than half the total entries. You don't want to discourage more than half of those who entered.

[Jacob W. Michaels]

Brother Fen wrote:

What might help people is perhaps a Junior RPG Superstar In-Forum practice competition where there would be open voting in a thread where everyone posted their items and then offered feedback. It could perhaps help people learn to accept criticism and rejection.

I'd call that the Blazing 9 thread, which starts right after Superstar ends and runs the nine months until the next one begins every year. It's a great practice thread where you can post items and get good feedback. It's probably not that different from the homebrew forum, but I think the handful of people on it (including myself) really look at the items from a Superstar perspective, which changes things a bit. I highly recommend the thread and am always surprised it's not used more.

[Jacob W. Michaels]

I think Floyd makes some good points. I have to admit I stopped voting shortly after the cull. Not because of it, but I just hit my limit on voting. Truthfully, I think I'd rather see one less week of voting next year (though if Paizo thinks the extra week makes that big a difference to the rankings, obviously I'd deal with it). Perhaps I'm in the minority here, though, since we did have at least double the traditional number of champion voters. Still, a little less time voting would also probably cut down on some of the complaints in the ramble thread purely because there'd be less repetition (though I think that thread honestly hits its maximum output fairly early and then starts to slow down).

I don't think it'd affect anything if the Top 32 were announced today. The voting would have ended Jan. 6, meaning the judges would still have had time post-holidays to go through the top items.

[Nickolas Floyd]

GM_Solspiral wrote:

Thomas LeBlanc wrote:

Did you like the big cull in this year's contest? Vote yes or no by liking the appropriate post.

The ones that are so offended by the cull that they left... aren't going to vote, because they left.

I think this is somewhat right. I was culled, but I'm not "offended." And I'm still here, though I've only voted once since Friday cause I'm just not that interested if I know my item will not come up. I'll be interested again when the top 32 are announced and I'll vote in the other rounds, but I don't feel like picking through 350 magic items knowing I'm already eliminated.

I'm disappointed that I was in the bottom 450, but I didn't expect to make top 32 this year anyway. Is my dislike of the large cull somewhat sour grapes? Maybe. I am human. But my stance is that there are more than 400 who likely feel the same way. Many likely don't even pay attention to the forums to even know they were culled, but many of the more enthusiastic would be designers (often the better designers) do. I'm old hat at this and I'll definitely be back next year, but how many may not?

I do think the deep cull was likely a net loss for the competition, but I don't know that for a fact. Right now there are 23 who liked the yes (I liked both the no and the neutral). How many of that 23 were culled? As GM_Solspiral, said, they may have taken their ball and gone home. And that is a sad thing. If you were not culled, you likely like the deep cull and if you were culled you likely don't like the deep cull. This is especially true for those of us who are active on these forums. But is it overall bad or good? I fail to see what the positive effects the competition gains from a deep cull that it could not have gained from a smaller cull (or two small culls). It would not have shortened the total voting time. It would not have given the judges fewer items to go through (top 100 either way). And it would have disheartened fewer hopefuls and possibly retained more active voters.

It would have given fewer really close pairings for an individual voter I'm sure. But it would also retain close to a third of some peoples' pre-cull favorite items. Maybe some of these would rise to the top 100. Maybe not. But is there really a point to cutting them?

[Brother Fen]

I found it interesting watching the voting unfold real time over the holidays. First everyone was excited to start voting. Then they voted and voted and voted. And then the complaining started. And compaining and complaining and complaining. Calls for an immediate cull went out with impassioned pleas. Then the cull happened. And the complaining resumed - except now everyone was upset that they were part of the cull.

I say cull 'em all and let Paizo sort 'em out.

If they left, then they needed to leave.

[Clay Clouser]

Culls allow paizo to spread the pack out, by cutting the bottom x% those items no longer are getting paired with the good items that get an auto vote just because the other item is not very well done, or isn't formatted, or is a joke item. Post cull you take a group of relatively closely clumped items and force them to be separated out by being compared only to each other. If you want to just don't look at the items I've seen thread and you will never know if you were culled or not. The contest isn't to see how many people paizo can get to vote. It's to find the best designer. If the culled people stop paying attention after the cull, that's sad because it means they wont get better. But it is still good for the contest over all.

[Brigg]

Nickolas Floyd wrote:

GM_Solspiral wrote:

Thomas LeBlanc wrote:

Did you like the big cull in this year's contest? Vote yes or no by liking the appropriate post.

The ones that are so offended by the cull that they left... aren't going to vote, because they left.

I think this is somewhat right. I was culled, but I'm not "offended." And I'm still here, though I've only voted once since Friday cause I'm just not that interested if I know my item will not come up. I'll be interested again when the top 32 are announced and I'll vote in the other rounds, but I don't feel like picking through 350 magic items knowing I'm already eliminated.

I'm disappointed that I was in the bottom 450, but I didn't expect to make top 32 this year anyway. Is my dislike of the large cull somewhat sour grapes? Maybe. I am human. But my stance is that there are more than 400 who likely feel the same way. Many likely don't even pay attention to the forums to even know they were culled, but many of the more enthusiastic would be designers (often the better designers) do. I'm old hat at this and I'll definitely be back next year, but how many may not?

You know, I know exactly how the culled feel about this.

Picture: You're doing votes after being culled, and every entry you see with the littlest mistake just makes you think, "Why did this one stay, and mine got culled?" Some people get the feeling that they can't stand to wade through other people's victories.

This thought pattern rings very, VERY deep with me. Back in June, a five-year relationship with my then-fiancee was abruptly cut off. Now, I can't stand hearing people talk about their wonderful relationships with their spouses, and I can't even stand the idea of anyone even inviting me to a wedding. I wouldn't be able to handle it.

I was culled; cast aside.

Why do I have to wade through everyone else's success?

So yeah, it sucks. And now that I think about it, I can't help but think I'd wane in my voting frequency if I had been culled. My jealous tendencies are an unfortunate personality flaw that I'm trying to work on. I know I'm prepared to be told I'm not in the top 32, but I'm still having high hopes.

[Kris Newton]

Most of what we're debating in this thread is the (potential) loss of voters after a deep cull vs. the greater ranking precision that we get from voting on a smaller batch of items. That debate's pretty fruitless to have here: Paizo no doubt has the numbers that will show who's right. If Paizo knows how many unique voters they lost post-cull and how much the smaller field helped sort the remaining items, they know whether the deep cull was a good idea.

I'm not sure we should strive to placate voters who are going to stop participating as soon as they're eliminated, given that we're always going to lose them after the Top 32 are announced. However, if retaining those voters as long as possible is really important, maybe Paizo could keep the deep cut but reinstate something like the golden ticket: say, each judge picks one culled item to save and consider alongside the Top 100, and those items are revealed at intervals during the post-cull voting. There would be no harm in revealing and discussing these choices publicly post-cull, because the tickets would already have been assigned and the culled items would no longer be up for voting.

This would put a public focus on items that won't necessarily make Top 32, which is something we've avoided in the past; usually, only the Top 32 are open for public critique unless requested by the designer. However, we've all signed up to take public criticism if we make Top 32, so I think most of us would be willing to accept public critique (retaining our anonymity, of course) in exchange for a golden ticket.

[Thomas LeBlanc]

I liked the big cull this year, as afterword the voting was much better than after the final culls of the previous years. What I would really like to see is keeping the 4 week voting period, but a 20% cull after each week of voting.

[Anthony Adam]

Or even in the last week, culling to the remaining Top 100 - evil grin :P

[Cthulhudrew]

OwlbearRepublic wrote:

However, if retaining those voters as long as possible is really important, maybe Paizo could keep the deep cut but reinstate something like the golden ticket: say, each judge picks one culled item to save and consider alongside the Top 100, and those items are revealed at intervals during the post-cull voting.

Note that Adam Daigle said in another thread that as far as he knows, the Golden Ticket is still a thing.

[frank gori]

Cthulhudrew wrote:

Note that Adam Daigle said in another thread that as far as he knows, the Golden Ticket is still a thing.

I feel like that should be discarded as it is a relic of the 3 judge system. I think the items should b selected from the top 100 or so period. If I understand the golden ticket correctly the judges can basically overrule everything thus far and resurrect a culled item if they wish it.

Some of the culled might even deserve it but it's kind of like technically the electoral college in that technology has changed we should prolly discard this.

[Thomas LeBlanc]

I like the concept of a Golden Ticket. Mark said he had 40 items on his keep list prior to the cull. 7 items on it were culled. What if he really liked an item and wanted to keep it? He is a judge and his opinion matters (just like everyboby elses). If he wants to keep an item, he should be allowed to. Maybe he saw something in the item design he thought showed signs of greatness, but was cut due to lack of a formatted template. I think any judge should be allowed to keep an item they think is Superstar.

Opinions vary, what we think as the voting public often doesn't match up to the judges selections. But the contest still works out to produce great results. The role of the judges are to look deeper into what an item's designer did. If they see something they like, they should be able to keep it.

Looking over the exit poll so far, about half of my keep items only have 2 votes. The voting public often vote with a bias. I see items near the top of the poll with (what I think) are horrible rules usage, vastly out of whack pricing, or just plain boring. Again these are my opinions and the poll shows they are not popular opinions. As with every year, if blood is mentioned once, the item most likely won't make the Top 100 according to the public, but may be a Superstar idea.

[Mark Seifter]

There are only three items this year that were good enough that I would have considered golden-ticketing them out of the cull, but they were good enough that they also survived the cull. The role of public voting is to help the cream rise to the top, since year after year we have more entries, and the job of looking through everything and choosing a Top 32 was becoming more than a panel of judges could possible bear. It also has other positive side effects like increasing everyone's awareness of good and bad things to do in item design when they have to judge items against each other. Out of those three items, there's only really one that I would want to golden ticket no matter what, but for the same reasons that I would do so, I also think it won't need it.

[Alexander Bennett]

Well, the cull didn't encourage me to vote anymore, but I was already pretty dead from before. I'm just trying to get myself to pay attention to the entire contest this year (which I haven't been able to for the last two years).

[frank gori]

Mark Seifter wrote:

There are only three items this year that were good enough that I would have considered golden-ticketing them out of the cull, but they were good enough that they also survived the cull. The role of public voting is to help the cream rise to the top, since year after year we have more entries, and the job of looking through everything and choosing a Top 32 was becoming more than a panel of judges could possible bear. It also has other positive side effects like increasing everyone's awareness of good and bad things to do in item design when they have to judge items against each other. Out of those three items, there's only really one that I would want to golden ticket no matter what, but for the same reasons that I would do so, I also think it won't need it.

I'd put money on it that it was a ring and it had a tail to tell.

[Joel Flank]

I just wish the cull had happened earlier - for about a week pre-cull I stopped voting, but have gotten some more votes in since then now that I don't have to suffer through some of the bottom of the pack (and it has let me see a lot more new items than it seemed I was seeing pre-cull.)

[Chris Shaeffer]

Maybe it's because I've been busy this year and not participating in the banter as much, but I get the impression that communication from the Paizo/judge end has been pretty slim this year so far.

I feel like smaller and more frequent culls along with more frequent main page updates would do a lot to keep the competition seem fresh several weeks in. It would garner more active participants and enrich the community.

Updates like judge interviews (articles or videos), official statistics, or teasers about future themes and twists would be cool.

In terms of statistics, I would like to see:

A simple list of all items entered (maybe wordcounts to distinguish identically named items)

Number of total votes (not for items individually)

Top voters!

Things like that. Maybe top voters could get a little prize?

--

On golden tickets: I think it's good to let judges reserve a few spots for their favorites. Unlike us, they are able to save really good items that might have fallen in the cracks, and it injects a little bit of "wildcardness" and excitement into the game.

[Chris Lambertz]

Just popping in to say thanks for this feedback. I'm making notes from this thread to pass along :)

[Nickolas Floyd]

I was voting fairly sparingly prior to the cull, expecting to vote more after the really bad items were dropped (no offense to those designers, it took guts just to enter and I'm down there with you this year, learn from it and come back strong next year). I did not see my item prior to the cull, but I was not worried about it expecting that it would be in the top 75% or so and I'd have a better chance to see it after the cull. I was fairly certain I would not make the top 32 this year. The item type switch really took the winds out of my sails and I did not have any confidence in the item I submitted. But I did not expect such a large cull and to be cut. I voted a lot the day after the cull not accepting that they culled so many without announcing that fact and that mine did not make it. I also saw a few items I could not believe were not cut. I reached Dedicated that day but stopped when reality set in and items added to the list slowed to a crawl (I added seven or eight items to the list that day).

Next year I'll be ready for any type of magic item and be confident in my entry (and they'll switch it to spell on me).

[Jacob W. Michaels]

I think Chris brings up a good idea about the greater communication being a good idea. I think having the cull announced more formally -- and maybe an explanation of why it was so deep (especially since it wasn't so deep in previous years) -- may have helped alleviate some ill feelings?

I definitely like the teasers we've gotten in previous years (i.e. the number of each type of monster before Round 3) and hope those continue this year.

[Brigg]

Nickolas Floyd wrote:

Next year I'll be ready for any type of magic item and be confident in my entry (and they'll switch it to spell on me).

Plot twist: Next year's round 1 will be "Create a Monster and Stat Block"!

[Isaac Volynskiy]

I'm holding out for "Create a magical fruit or vegetable".

[Brigg]

"Create an expendable Wondrous Item!"

[Chris Shaeffer]

"Create Backstory for Wondrous Item"

[Thunderfrog]

"Create a mundane item!"

[Woody Elliott]

"Create a filigree for a magical item"

[bugleyman]

Thunderfrog wrote:

"Create a mundane item!"

Dibs on the Red Swingline Stapler.

[Clay Clouser]

bugleyman wrote:

Thunderfrog wrote:

"Create a mundane item!"

Dibs on the Red Swingline Stapler.

You broke anonymity. you are DQ'd

[Sir William]

The golden ticket concept makes sense to me. It isn't a guarentee that the other judges will like the item, but it gives the possibility that a subtle item (perhaps with a mechanic that was overlooked) can get in front of the judges.

[Brigg]

"Submit an Adventure Proposal"

[R D Ramsey]

Create a nation of intelligent blinkdogs.

(still one of my favorite moments)

[bugleyman]

Clay Clouser wrote:

You broke anonymity. you are DQ'd

Dammit!

[CripDyke]

Meh. I didn't have any items in this year's contest, but I voted.

As someone who wasn't in it just to have my own item upvoted, I deeply appreciate the cull so as not to have to deal with truly unqualified entries. Waiting 60 seconds to vote between an item with no bolding and little formatting and nothing new (a staff with existing spells and no other powers, for instance) and something with some level of attention to detail behind it is a waste of everyone's time.