When is the cull?

[Mark Griffin]

Based on previous years, how long until the first cull? Do they generally do more than one?

[Feros]

It's likely to occur either Tuesday or Wednesday. Enough time to really separate the cream from the dregs.

[Cthulhudrew]

Yeah, I'd say it's probably going to happen after a week, given the time frame of this round. There was a longer voting period last year, but that was largely because they weren't sure what kind of turnout they might expect (from at least one report).

Given the overwhelming turnout- and the massive effort by a couple of individuals (*cough, cough* Feros *cough*), they shortened the voting this year, and will probably make the cull pretty quick. I kind of doubt there will be more than one, but who knows.

[Feros]

Hey, I suffered alot last year! They placed that cull just at a point where I had nearly lost all hope! (As the voting advances, you see quality items less as they get up-voted more. It got to be a bad scene after two weeks. *Shudders*)

[Chris Shaeffer]

Maybe it's because I started voting a little late, but I have seen very few items that I thought were up to scratch. I have noticed that items of similar quality tend to get paired up though.

From what I understand, the more people agree on an item (either voting for or against it) the less it shows up? It's the contentious items that are seen the most. Correct me if I'm wrong.

If that's so, even though a lot of the items I've seen are "meh", they aren't eye-gougingly bad. The overall quality of this year's competition is way, way up. :)

Haven't seen my item yet. :/

EDIT: Just saw one I really liked. :D

[Papasteve08]

Chris Shaeffer wrote:

EDIT: Just saw one I really liked. :D

.

Now 1200 people desperately want to ask you if it was their item you just saw.

Thanks for the teaser. :)

[Sean K Reynolds]

Just remember:

Assuming there are 1,000 items,
And only 32 make it into the Top 32,
That leaves 968 items that won't make it into the Top 32,
Which means a lot (over 95%, in fact) of not-good-enough items which need to be voted on to contribute to the sorting.

=====

Also, people agreeing on an item (by tending to vote it up or down) doesn't tend to make it appear less or more in later pairings; the pairings are random. Each vote overall adds to the sorting of the items, and the combination of all the vote pairings creates creates a web-like hierarchy, with "downvoted more often" at the bottom and "upvoted more often" at the top.

For example, in last year's R1 public voting, Ross did a preliminary ranking math-crunch on all of the items.
* The lowest-ranked item was voted on (up + down) 588 times.
* The highest-ranked item was voted on (up + down) 586 times

* The most votes (up + down) any item got was 600.
* The least votes (up + down) any item got was 542.

Given that we had over 500,000 votes last year, the difference between 542 votes and 600 votes is statistically insignificant (in other words, another ten votes up or down for any particular item isn't going to push it far in either direction). Looking at the trends for the voting results, there's a clear hierarchy established by hundreds and hundreds of votes.

[Feros]

Yay! Mathematics to the rescue! :)

[Ignotus Advenium]

Sean K Reynolds wrote:

For example, in last year's R1 public voting, Ross did a preliminary ranking math-crunch on all of the items.
* The lowest-ranked item was voted on (up + down) 588 times.
* The highest-ranked item was voted on (up + down) 586
.

I wonder if the highest-ranked items did better than 90% / 10% up/down, and if the lowest-ranked did worse than 10% / 90%.

[Lightminder]

If it wasn't for the unkindness of public forum negative critiques, it would be neat to take the worst three items and get the winners to crit and rework them.

[Chris Shaeffer]

Yeah, I was noticing the negativity is quite rampant this year...

[Sean K Reynolds]

Ignotus Advenium wrote:

I wonder if the highest-ranked items did better than 90% / 10% up/down, and if the lowest-ranked did worse than 10% / 90%.

It was closer to 75%/25%, at least for the top few items.

[Cthulhudrew]

Chris Shaeffer wrote:

Yeah, I was noticing the negativity is quite rampant this year...

I don't see most of it as negative so much as playful snark.

[Landon Cole]

I think a few people have been taking part in RPG SuperStar for a while and are very familiar and comfortable with it. SIAC, SAK and so on are well understood. They see a lot of items that fall foul of the advice without, in their opinion, meeting the 'you can ignore the other rules if you're item is great' catch-all.

I think a new person, who's entered for the first time, could take the snark (perhaps legitimate, if the rules and advice haven't been read) not as a criticism of the item, but as a criticism of themselves. It's not hard to work out which item is being disparaged in some cases, or it could be one of a small number. It might be meant as a playful snark, but could be taken as a bit of a kick in the teeth.

The ultimate aim of RPGSS is to find new writers for Paizo, but I'd hope it's also to encourage everyone to try their hand at writing. The first time people enter, their items might well suck, but they might improve into a really good writer whose work we all enjoy and appreciate - with a little encouragement. A snarky comment might be enough to put them off trying again in future.

A few people have said this on the boards, but I think we do need to be a bit less specific in describing items we've seen and a bit more forgiving of mistakes.

[Tokoz]

So, if I'm understanding Sean correctly, there is no cull this year because last year's numbers show it's unnecessary?

Does this mean I have to get back to voting again instead of waiting?

[Thomas LeBlanc]

Landon Cole wrote:

The first time people enter, their items might well suck, but they might improve into a really good writer whose work we all enjoy and appreciate - with a little encouragement.

Yes, don't be discouraged! My first item was horrible and my second year I got a spiffy Top 32 tag.

[Maurice de Mare]

Well the cull might not happen if we keep op the momentum of voting, if however the momentum stops Paizo can trigger a surge by colling.

Everyone will need to check if their item is still there.

[Monica Marlowe]

Lightminder wrote:

If it wasn't for the unkindness of public forum negative critiques, it would be neat to take the worst three items and get the winners to crit and rework them.

That is a very unfair statement. I believe once we are given our Top 32 you will see a lot of very helpful comments and kindness pour out of the many people that you believe are currently being unkind. While it is hard to assess tone and intent via the internet, I can assure you that the vast majority of individuals venting are just doing so because they are in a place where they cannot speak their reactions out loud, that does not make them mean or cruel, just people who need a pressure release.

If you find my assessment of my fellow voters to be wrong on January 22, please let me know. I would very much like to make amends.

[Sir William]

(@ Lightminder) When I started looking through the threads, as someone new to the message boards, there were some "ouch" moments for me. While I think it is understandable to momentarily conclude the forums are "unkind," I think that Mamaursula is correct.

Multiple people have gone out of their way to assist the me (the newbie) in just the past week. One sent me a private message because were worried I was going to say something specific about my item. Once people can help (their hands are tied right now), I expect that the focus will change.

[Jacob W. Michaels]

Landon Cole wrote:

It's not hard to work out which item is being disparaged in some cases, or it could be one of a small number. It might be meant as a playful snark, but could be taken as a bit of a kick in the teeth.

This is my problem with the rambling thread. I don't mind people groaning over poor formatting or dull items, but let's not identify which ones we're talking about so clearly. I don't think I should be able to figure out which item people are talking about when I get to them. It's not fair to the competitors, IMO. (I don't remember what the "It's a trap" item is off the top of my head, for example, but I wonder if all the comments that have clearly identified it have people downvoting it without even reading whether the item itself is good or not. Or socks may have been an odd choice, but maybe someone likes that item in spite of that but feels he/she shouldn't because of the comments that are clearly about it.)

[Rich Malena]

Jacob W. Michaels wrote:

This is my problem with the rambling thread. I don't mind people groaning over poor formatting or dull items, but let's not identify which ones we're talking about so clearly. I don't think I should be able to figure out which item people are talking about when I get to them. It's not fair to the competitors, IMO.

There are also a few unfortunate criticisms. I groan whenever anyone mentions items with mismatched pricing, since two items that I know of have clear and obvious typos that the ramble thread has interpreted as authors who don't understand how to price items. Or at least that's how I read those posts.

A general post that said "Bob Barker is very sad about these incorrect prices" is one thing, but a post that directly copies the prices (WHICH CURRENTLY EXISTS) just begs folks to vote against it.

Likewise, the oddly specific Bingo game in the thread. Funny joke, but has it incited new voters to auto-downvote those items?

These are inherent problems with public voting (see: East Coast exit polling's effect on West Coast voting), so I don't think much can be done about it, gag order or no....

Sorry, we're talking about the cull. Hey, when it happens, can we also cull these threads? Lets get back to the vague praise!

[Sean K Reynolds]

There will be a cull this week. Doing so helps the voting process by allowing the votes to focus on the better-quality submissions, and helps prevent voter fatigue by making sure the voters don't see the lowest-quality items any more. :)

[TimsterMon]

Thanks for the "head's up" Sean! 8')

[Lightminder]

hey next year can it be set up to show how many votes you have placed so we can budget for goals, set goals for ourselves etc...

[Sean K Reynolds]

That's a question the tech team can answer (I don't know how much work it would be to add that, and it's not my place to obligate them).

[Cthulhudrew]

Sean K Reynolds wrote:

That's a question the tech team can answer (I don't know how much work it would be to add that, and it's not my place to obligate them).

Yes. Nothing builds workplace resentment like having other coworkers volunteer you for projects.

[Thomas LeBlanc]

Lightminder wrote:

hey next year can it be set up to show how many votes you have placed so we can budget for goals, set goals for ourselves etc...

Back in my day we made little marks on paper—called "ticks"—to keep track of how many times we had done something. There are also mechanical talliers for those so inclined. Combining either of those two methods with a calendar and some maths should suit your needs (without distracting the Tech Team from finishing up the Paizo Game Space).

[Tokoz]

Thomas LeBlanc wrote:

Back in my day we made little marks on paper—called "ticks"—to keep track of how many times we had done something. There are also mechanical talliers for those so inclined. Combining either of those two methods with a calendar and some maths should suit your needs (without distracting the Tech Team from finishing up the Paizo Game Space).

Yes. Old School. Although it might be something as small as adding an extra call on the master page. The database already has some numbering it's doing. I'm sure Gary will know the difference between that and something that will seriously hamper Game Space.

Can't hurt to ask.

[Lightminder]

doi yup. will tick the paper then.

[Monica Marlowe]

Thomas LeBlanc wrote:

Lightminder wrote:

hey next year can it be set up to show how many votes you have placed so we can budget for goals, set goals for ourselves etc...

Back in my day we made little marks on paper—called "ticks"—to keep track of how many times we had done something. There are also mechanical talliers for those so inclined. Combining either of those two methods with a calendar and some maths should suit your needs (without distracting the Tech Team from finishing up the Paizo Game Space).

You probably had to walk up hill both ways in the snow to go to school every day in the dark too, didn't you? ;-)

[Thomas LeBlanc]

mamaursula wrote:

You probably had to walk up hill both ways in the snow to go to school every day in the dark too, didn't you? ;-)

Hell no! When it was dark out, we had to feed enough animals to fill an ark. We also had a bus, which we had to push the bus to school uphill and then strain our backs pushing against it so it didn't roll too fast downhill.

[Lightminder]

I lived on the prairies and there was no up hill, just below zero- 50 below with a high of minus 39, and if your hat blew off you had five minutes to get home before your skin started to die. putting meat on the porch to freeze, boots in the fridge to keep them from thawing. (takes out teeth gives them a shake and re-fits them into mouth)

The styrofoam squeak of boots in snow and the sight of frozen pets we wish we had a bus to push, the effort would have kept us warm. We hear southerners cancelled school if it even snowed but i never had a day of school cancelled.
(leans back in rocking chair so hard it gets stuck under the mantle- he sees what has happened and acts like he MEANT to do that)

No one guessed your weight they all talked about surface to mass ratios and thawing times, scientists would be the guys jumping into the river and then timing how long to get them back to breathing and bleeding and room temperature using different methods. foreign militaries training for Siberia came here because we had the same climate, even though our population was mostly Mennonites, Hutterites, Doukhobours and Quakers at that time (especially around the town of tolstoy).
( gestures with thumb to an axe mounted over mantlepeice, used at clear lake Conscientious objector work camp)

tires would shatter if they were even a little flat and that was the season to sell riverfront property to southerners from Toronto, because they couldn't believe this frozen wasteland would flood in the spring, then be an entomologists dream when the floods subsided, ah but autumn, there was a nice week, but dammit we had digital vote counters from diebold!. (strikes match and lights pipe full of dusty tobacco from a tin bought on sale back in 1956, he is down to his last eleven tins, the house is made of the rest embedded in cobb and covered in tar paper and thousands of old sand bags now UV deteriorated and blowing in the wind like spent liverwurst casings, a branch is stuck on the sod roof, deposited by last years flood)

[Voice Of Reason]

Feros wrote:

It's likely to occur either Tuesday or Wednesday. Enough time to really separate the cream from the dregs.

I imagine that we won't be informed if our item makes it or not. Am I correct in thinking that the only way to know if we've made the first cut is to keep voting and hope to see our item after that point?

[Eric Morton]

Arkos wrote:

Likewise, the oddly specific Bingo game in the thread. Funny joke, but has it incited new voters to auto-downvote those items?

These are inherent problems with public voting (see: East Coast exit polling's effect on West Coast voting), so I don't think much can be done about it, gag order or no....

I can't speak for other voters, but I judge the items I'm seeing based on the items themselves, not other voter's commentary. While it's true that I'm entertaining myself with a scavenger hunt for the infamous items people have mentioned in the rambling thread, that hasn't somehow compelled me to abandon my own judgment and conform to the opinions of others.

[Eric Hindley]

Eric Morton wrote:

Arkos wrote:

Likewise, the oddly specific Bingo game in the thread. Funny joke, but has it incited new voters to auto-downvote those items?

These are inherent problems with public voting (see: East Coast exit polling's effect on West Coast voting), so I don't think much can be done about it, gag order or no....

I can't speak for other voters, but I judge the items I'm seeing based on the items themselves, not other voter's commentary. While it's true that I'm entertaining myself with a scavenger hunt for the infamous items people have mentioned in the rambling thread, that hasn't somehow compelled me to abandon my own judgment and conform to the opinions of others.

I know I'm in the same boat. But I suspect newer voters might be more likely to be swayed by the negative comments of others, especially a perceived bias against the "auto-rejects" and bad formatting issues. It's easy to read a lot of comments that say such-and-such obviously easy to identify item is bad, and let it sway your vote. They see an item that breaks one of the unwritten rules of the contest heckled for a bit and decide that they should vote it down. I've seen a few too many X vs. Y posts in the long thread not to believe this is happening. The Auto-reject post does have a rule 27 you know...

[Curaigh]

Voice Of Reason wrote:

Feros wrote:

It's likely to occur either Tuesday or Wednesday. Enough time to really separate the cream from the dregs.

I imagine that we won't be informed if our item makes it or not. Am I correct in thinking that the only way to know if we've made the first cut is to keep voting and hope to see our item after that point?

Even if they did, I would will have to see mine to feel comfortable :)

As far as not swaying votes, keep in mind SiaC & SAK were terms invented for RPGSS & you had to develop your 'tastes' somewhere. I have seen multiple things quoted as auto-reject in that thread that aren't one of SKR's design traps (I stopped reading 4k posts ago fwiw).

[Rich Malena]

Eric Morton wrote:

Arkos wrote:

Likewise, the oddly specific Bingo game in the thread. Funny joke, but has it incited new voters to auto-downvote those items?

These are inherent problems with public voting (see: East Coast exit polling's effect on West Coast voting), so I don't think much can be done about it, gag order or no....

I can't speak for other voters, but I judge the items I'm seeing based on the items themselves, not other voter's commentary. While it's true that I'm entertaining myself with a scavenger hunt for the infamous items people have mentioned in the rambling thread, that hasn't somehow compelled me to abandon my own judgment and conform to the opinions of others.

And that's fine. But I'm a teacher. I don't worry about what I would do. I worry about what other people would do, and use that to draw a line.

On another note, infamous implies negativity. If you're thinking about them as infamous, doesn't that affect your judgment?

[Eric Morton]

Eric Hindley wrote:

But I suspect newer voters might be more likely to be swayed by the negative comments of others, especially a perceived bias against the "auto-rejects" and bad formatting issues.

I have a bit more faith in first-time voters than that. I trust that they know a really cool item when they see it, and are savvy enough to realize that editors can make minor fixes to an item if that's all that's holding it back.

[Eric Hindley]

Eric Morton wrote:

Eric Hindley wrote:

But I suspect newer voters might be more likely to be swayed by the negative comments of others, especially a perceived bias against the "auto-rejects" and bad formatting issues.

I have a bit more faith in first-time voters than that. I trust that they know a really cool item when they see it, and are savvy enough to realize that editors can make minor fixes to an item if that's all that's holding it back.

I don't doubt that many of them can, but I still think that seeing the words "auto-reject" and "repeating themes" and "oh no, not this again" is bound to stick with people new to the contest. I think Sean probably regrets calling his list auto-rejects, to be perfectly honest.

[Feros]

There are also a lot of voters who don't even bother to read these threads. The truth is we are not representative of the whole.

[Eric Morton]

Arkos wrote:

On another note, infamous implies negativity. If you're thinking about them as infamous, doesn't that affect your judgment?

Nope. Richard Feynman was an infamous ladies' man. He was also one of the greatest physicists to have ever lived. His infamous reputation does not prevent me from recognizing his intellectual achievements.

[Rich Malena]

Eric Morton wrote:

Arkos wrote:

On another note, infamous implies negativity. If you're thinking about them as infamous, doesn't that affect your judgment?

Nope. Richard Feynman was an infamous ladies' man. He was also one of the greatest physicists to have ever lived. His infamous reputation does not prevent me from recognizing his intellectual achievements.

Sure, and I completely agree. And if this was a closed community of folks who also agreed, then that would be fine. But this is public, and my grandmother would absolutely disagree about Feynman.

In my physics class, I'll present Feynman's lectures, and I might even hint at his character, but I won't highlight it because I don't know how individual students will react and I don't want it to derail us from his achievements. Just because I think one way doesn't for a second mean I believe that everyone will agree.

Anyway, clearly we disagree, and I'm tired of arguing, so back to voting.

[Samuel Stone]

How much is usually culled? Bottom 25%?

[Feros]

Samuel Stone wrote:

How much is usually culled? Bottom 25%?

Well, this is only the second year for voting, so we only have the cull from last year to consider. But yes, I believe the SKR or one of the others said somewhere on the boards that it was the bottom 25%.

[Brian L Clark]

Sean K Reynolds wrote:

Just remember:

Also, people agreeing on an item (by tending to vote it up or down) doesn't tend to make it appear less or more in later pairings; the pairings are random.

Oh, thank you for that. I haven't seen mine yet and I was starting to be really concerned.

[Sir William]

Eric Morton wrote:

Eric Hindley wrote:

But I suspect newer voters might be more likely to be swayed by the negative comments of others, especially a perceived bias against the "auto-rejects" and bad formatting issues.

I have a bit more faith in first-time voters than that. I trust that they know a really cool item when they see it, and are savvy enough to realize that editors can make minor fixes to an item if that's all that's holding it back.

This is my first time as a voter. Somewhat new to the system, this is the first time I have followed the "Incessant" threads. While they have informed me of "things to avoid" for the next contest, if they have any influence on my vote, it is to give more leeway to minor editing problems (the judges can take care of that). Sean's informational posts (super-helpful) seemed obviously not really about "auto-rejects," which I think is repeated quite a few times.

(This) New Voter's Process:
1. Read it.
2. Pick my favorite if it is obvious.
3. If it isn't clear (because I don't know a rule) then I check the rule.
4. If it is still unclear I will not vote (has happened maybe once or twice)
5. If it isn't clear (because they are both good/bad) then format is a tie-breaker.

New voters that may be affected I suspect will not vote 100+ times. People taking that much time to vote (hopefully) will be those reading everything and then forming an opinion. I think most that might be swayed may have voted down those type of issues regardless.

[Kenton Abel]

For last year, I wonder if the items in the top 200 were in the top 200 at the first cull? Were the top 89 the only items with a positive vote?

If they set a cull to items close to positive, it might not make any difference to the final 32 since the judges pick those. I think I remember that an item in the top 10 votes was not selected by the judges last year. It would certainly make voting much more challenging.

[Aaron Miller 335]

Arkos wrote:

Jacob W. Michaels wrote:

This is my problem with the rambling thread. I don't mind people groaning over poor formatting or dull items, but let's not identify which ones we're talking about so clearly. I don't think I should be able to figure out which item people are talking about when I get to them. It's not fair to the competitors, IMO.

Likewise, the oddly specific Bingo game in the thread. Funny joke, but has it incited new voters to auto-downvote those items?

I have up voted many of the bingo items, It's just they do kinda stand out. I have found some people being to specific in rambling thread including exact prices, that needs to be addressed.

[Jacob W. Michaels]

Kenton Abel wrote:

I think I remember that an item in the top 10 votes was not selected by the judges last year. It would certainly make voting much more challenging.

Correct. In fact, it was item No. 3 if I recall. I felt really bad for that person (and really curious what the item was).

[Chris Shaeffer]

Sir William wrote:

(This) New Voter's Process:
1. Read it.
2. Pick my favorite if it is obvious.
3. If it isn't clear (because I don't know a rule) then I check the rule.
4. If it is still unclear I will not vote (has happened maybe once or twice)
5. If it isn't clear (because they are both good/bad) then format is a tie-breaker.

Yeah, I started with a more technical way of determining which one to vote for, but it pretty much ended up boiling down to something similar:

1) Read/refresh my memory on the items.

2) Follow my gut. This is more often a vote against something I dislike than for something I like, but after a few hundred votes I've found that I have immediate, visceral reactions (positive or negative) to many of the items.

Causes for dislike are usually poor word flow or over-discriptiveness which causes me to struggle through reading the entire entry, too much power, too many powers, obvious lack of rules knowledge, or lack of excitement.

3) When both are equally good, okay, mediocre, bad, etc... Yup, formatting. Formatting is barely even a factor until I reach a tie (unless it's an eyesore).