Search Posts
We've continued to discuss it -- just last weekend, in fact. We came to some strong conclusions about how we would have to do it (it's not as simple as having a contest, because the prize was obviously a strong part of what made Superstar what it was), but laying the groundwork will likely mean the contest itself wouldn't start until next year. Sounds like there's another reason I'm sad I missed PaizoCon this year! If anyone ever needs a pedantic game theorist to look at voting systems, you know where to find me. I randomly came to check out this forum today and saw that it had been shelved under "Older Projects." I was somehow hoping for more RPGSS fun in the future.
The Afghanistan Principle is not without its detractors or its problems. And obviously, there's a very popular roleplaying game that doesn't (and basically can't) follow it completely. Even so, it's a design choice I absolutely appreciate in both board games and RPGs. And I'll have to remember it when I'm sorting out my PACG box for later adventures. Great post!
I'm certainly not a frequent poster on the boards, but I have greatly enjoyed watching this entire competition. For what it's worth, there wasn't a single round where I didn't wish I had more votes to spend on amazing ideas. This final round is no exception! Great work to every one of the competitors and congratulations on getting a field of folks to read all about your ideas. If nothing else, that's absolutely worth it.
Rather than simply being an encounter, this is clearly a midpoint in a larger plot arc. And a cool plot arc at that! This isn't just a random one-off. The party is here for a reason, and I think that you've done a lot to develop some eeriness and weirdness in this moment. I see that you had shout listed as the link for your first haunt, and that would have given those two haunts some meeeeean synergy. The creatures are then kind of an obstacle to deal with while the haunts are trying to turn folks to stone and tear them apart. The change to crushing despair turns the party back towards the enemies, who are likely a little weak to be a real threat. Which is sad, because I really liked where the ruthlessness was going! I spent a long time considering where my second vote would go, and this encounter made it ahead on the strength of the larger storyline and plot. Great work!
I really like that this encounter is actually more than just one combat. There's potential preparation and a plan to ambush a devious and deadly foe. I'm a huge fan of giving player's choices during encounters, so this hits home for me! Learning about the bridge and using it as part of the trap... so much fun. Let the chaos begin. I'm worried that players would choose to make this event happen during the day, since that seems so much easier. I would try to avoid that, and make the stalling tactics a method of getting more time to ambush. There could be a solid mini-game there rather than allowing dawn to totally make this easy. This encounter was an easy choice for one of my two votes. Great work!
Seems like less folks are commenting this round. I wonder if they're waiting to play the encounters first or if the encounter round is just less exciting for people than the previous ones, since they have to really experience them to judge them. I will fully admit to being an inverted pyramid in this competition. I enjoy talking and posting and all the rest early on, but at this point, it's serious work if I want to be able to give worthy feedback. I've read through each encounter, but now I'm in theoretical playtest land to figure out where my votes go. Sorry for the silence, Top 8! The submissions are great!
Just gotta say that it's tough to make a small fey that throws damage around. So you need to find a new way to make them a threat to PCs, and I feel like your bestial transformation plan is a fun way to make that happen. This creature feels well suited to a long-term, strike-and-flee style action against the players, slowly sending them through a series of bear traps, and attacking from range before hiding in the woods. A pair of these with a GM willing to play them Predator style would make for a fun and memorable encounter. Anyone who put these into a regular Pathfinder meat-grinder would feel them to be a less than successful encounter. But that's the way fey seem to be designed at these levels. I would like to see more supernatural defenses to keep them at range, to make up for their low speed against something like fly. A few fun ranger spells may have helped out! I've got a known weakness for fey that require strategy from a GM, so you're getting one of my votes!
I just went through and my decision, and I thought those monsters were great! It's like the Top 16 is trying to make this a serious competition and stuff where I really have to decide what I value in a monster. I looked over the judges recommendations, but it's easier for me to think about what I want out of a creature than a map, so this one was basically all me. Anyway, if the Top 16 is looking BUT NOT POSTING, then I say great work, fearless designers! Huzzah!
The level of correlation between the judges' assessment categories and the results of the votes is astonishing. Not a single map advanced unless both judges recommended it. And all 16 maps that got recommendations from both judges advanced. I tried really hard not to look at the judges recommendations, but to instead make my votes personal. That being said, it was WAY too hard not to read those because they had such excellent reviews, and I ended up doing it more often than not. On the other hand, if selected RPG professionals didn't like a map, I felt like I had to come up with a pretty good reason why I liked it before I should even consider giving it a vote. That's basically the reason why they're judging, right?
You know, maybe it made its way into previews of the Round 2 rules which used the word flip-mat, but "flip" doesn't show up in the actual rules at all. But part of my voting process had been to look for things which I could use as a flip-mat in a huge pile of potential encounters. I'm going to have to run back through it before the round ends...
I am drawing my potential monster, in part to help me visualize it, and also because I want to show it off. Will I be able to use my drawing?Nope. Just make sure you don't post it until well after the voting period is over for the monster round! Otherwise it falls into that all-important rule #5 about adding information about your submission and all the rest...
Wow, I'm kind of hoping this thread gets deleted, since it doesn't add anything to the conversation. But just in case some of the competitors happen to check and see this before then, let me stop in and disagree with this post. Given a two day turnaround, on top of the fact that the map round uses a different skill set than the magic item round, I am not disappointed in any way. Rather, I'm EXCITED about all the amazing possibilities I see here. Everyone took a different approach to the round and we get to see what a talented group of designers think a map needs in order to be creative and useful. I had no trouble coming up with a list of favorites, and easily had to cut down to a final 8 votes. This is a great set of flipmats. My eight favorites deserve my vote in this competition! I guess everyone's entitled to their opinion, but still. I'm really sad when someone looks at 32 different works of creativity and feels compelled to post about how terrible they all are. Looking at the creative work of people only ever makes me feel positive. More positivity around here would go a long way towards supporting this competition. So, to answer your question, NO.
A really interesting conjecture with statistical evidence. DATA! I wish I could see it all. I really appreciate that you've noted a specific and quantifiable moment here. This is the kind of analysis that can really detail how the process is working with asymmetric voting patterns. I would love to see how those twenty votes compare to the total votes for that item, how that played out over time, whether high-volume voters have significant sway... all sorts of interesting questions show up! Why didn't I get into big data? Oh well.
Rich, I just want to clarify that the Items Seen thread did not provide voters with the ability to read the text of every single item. The tracking spreadsheet just provided the name and last reported seen date of each item.There was a document that the item seen spreadsheet linked to that contained someone's full-text list of items. That link was later removed. Ah, I missed that it was removed. I just remember that it was on the site early on the process.
I've got plenty of regrets in this one, though it did make it through all the culls. Silly name, never ending abilities, links to known spells... tear it apart, critics! Stormleaper’s Greaves
Once per day, the wearer may stormleap for up to ten rounds, bounding through the air as if from stone to stone. A stormleaper gains the ability to fly with a speed of 60 feet but the wearer uses their acrobatics skill in place of the fly skill to perform daring or complex aerial maneuvers. While stormleaping, the wearer is continuously surrounded by a thunderous aura, dealing 1d8 points of sonic damage to any creature within 5 feet unless they succeed at a DC 16 Fortitude save, though a creature may only take this damage once per round. A stormleap ends immediately once the wearer’s feet touch a solid surface. Once during a stormleap, the wearer may use a standard action to send the furious storm at a single target beneath their feet, dealing 5d6 points of electricity damage unless the target succeeds at a DC 16 Reflex save. When used outdoors during a storm, Stormleaper’s Greaves become significantly more potent, dealing 2d8 sonic damage instead of 1d8, and 5d10 electricity damage instead of 5d6.
You teach Game Theory, Rich? You're my hero. Awwww, shucks. :)
That is true. The likelihood of seeing workshopped items is overall greater than seeing a single item, but not by a ton. The larger problem is simply that a "workshop" is similar to a voting bloc and may have similar guidelines for how they vote during the competition. Even without a nefarious agreement, it does create a bias. Sorry if I implied something wicked happening! I was looking at the unintentional bias rather than some kind of plot!
Time to nerd out. RPGSS Open Call voting is done using the Schulze beatpath method, which creates a huge overall ranking. It's a modification of a series of preference-based voting methods, and is one that is currently used in many circles. Specifically, rather than a single ballot ranking all entries from best to worst, we get the pairing method we're all familiar with. Unfortunately, any kind of preference voting suffers from a few important flaws which are mitigated only under ideal circumstances. I feel as though some of these flaws are leading to the complaints we've seen, though I'm certainly not prepared to suggest any sort of major overhaul. I do, however, have some suggestions. Disclaimer: I voted often this year and in years past. I love the entries that made it in, though I certainly wanted to get into the Top 32 this year. I also teach Game Theory and have trouble not thinking about things like voting methods and strategic voting, and I’ve used this competition as a class analysis in the past. So while this critique may have some feels behind it, I also plan to make sure it has some solid math as well. Take it as you will. The Nitty Gritty:
Any preference based voting system generally fails to prevent Tactical Voting. On a ballot of five names with two major contenders, voting one of those contenders into last place in order to tank their chances of competing with the other contender is a solid example of tactical voting. The Schulze beatpath method claims to be resistant to Tactical Voting, since the tactics of a single voter on a single pairwise vote is outweighed by the "unpredictable" masses voting as they see fit.
"Unpredictable" is an important word. It implies that either everyone is voting tactically their own way, or that one voter cannot predict the votes of another. Unfortunately, the current state of the competition makes many votes more predictable than one might think.
The last years have avoided these pitfalls by allowing a group of judges final say over the list (which has been generated using these same pitfalls, but let's leave that aside for now). How else could we get public voting to a place where it can sincerely create a top 100 list?
Unfortunately, I think these pitfalls are unavoidable given the nature of the competition. I think that judges are one way to mediate these problems, by taking the final rankings out of the hands of a flawed public voting scheme. Unless that returns, we have the current method: a voting system where unintentional systemic biases lead to tactical voting. I also don’t suggest the judges are anywhere near as fair as the ideal Schulze beatpath system. I simply want to point out that we are currently doing all the things that make Schulze beatpath unfair. I would and could get more specific, but I feel like that sends me more into “sad also-ran” contention rather than being an analysis of the process. I just want to say again that this year’s process created a good outcome. I don’t see the Top 36 as unworthy of their place in any way. They are great representations of the good things that come from this competition. I'm actually hesitating to hit submit because I enjoy this competition so much. However, I do want to give some legitimacy to some of the complaints about this year’s process and to hope that some analysis leads to a change in how the process is done in the future. If you read this whole thing, you are my hero.
GUY #2: "Okay, fine, but I'm inviting my girlfriend. And we're stopping at my house so I can steal some of my Dad's beer. If I'm going to drive out to a big, empty house in the middle of the night, I'm going to have some fun while I'm there." So are you also going to find a Professor/Priest and a couple of kids? Or are we not playing Betrayal at House on the Hill? Because some Betrayal would really liven this place up!
In one of the two possible future realities, I will regret that I freed up tonight for work by moving my plans to tomorrow... But at least in both realities, I'll be well-rested for Tuesday! Yay, right?
I had to stop voting before the latest two culls, so I haven't had a great chance to see the future here! I'm really excited for the final reveal. Good luck, competitors! I'm also VERY VERY excited to see what data Paizo is willing to put forward about how this round went... I think the voting changes this season have been interesting overall. Let's see how they play out!
I will regret signing the papers to buying a house in the middle of the voting round if I end up in Top 32 and can't find the time to continue on.On the plus side, if you make the Top 32 and can't find time to continue on, you get the opportunity to make one of the alternates very happy. As one of those alternates who got the call, I'd call it 50% happy, 50% complete and totally stressful horror insomnia freakout.
First, I really want you to write another scenario! I cared about it enough to review it, and I hate that it's been taken over by knee-jerk one-star reviews. I happily ran it and I would happily so again. If you ever come down to the local PFS at Guardian Games, I'll bring the 3d terrain I built for the bluff because I am a crazy person. A comment was made on the Items Seen post discussing the desire to scour the cull data to look for key features that the crowd plays to. Price ranges. Item types. Keywords. And if we start playing to the crowd by designing items like that, then the overall nature of this competition is going to change. Instead of a design challenge, we have to add some marketing to our submissions. The knowledge that there are no golden tickets is a little scary. Some of the posts in this thread with blanket dislikes and downvotes worry me. My feelings are highly represented in Garrett's thoughts here. Of course, I still want to sway the crowd because I DO have imposter's syndrome. So I kind of want that data. I'm studying the snark thread. I'm conflicted.
30% of the grade sounds eminently fair! Now I'm going to have to come up with a grading rubric... There go my plans for today.Rich no, think of the children! Are you kidding? We're prepping for the school year this week. This is the kind of thing I spend far too much time doing! Anyway, for those interested in what the Teachers are up to these days, we're starting to see a lot of stuff like this. This isn't meant to be a serious document, and it certainly isn't how I'm actually voting, but if we really wanted to start grading entries... If I were to do something like this seriously, there would be multiple description categories for fluff and crunch, plus grammar and punctuation, plus plus plus...
Not saying it is an auto-down vote, just 30% of the grade. 30% of the grade sounds eminently fair! Now I'm going to have to come up with a grading rubric... There go my plans for today.
Going against the grain here... but, I'm a teacher. And if my unique little snowflakes don't indent their paragraphs, but include outstanding thesis statements, I'm still going to give them a high grade. Maybe not the highest, but still. I like to see a good template. I want to see all the pieces in the right place. I don't like it when important things—like construction requirements!—are missing, but I'm not going to instantly downvote a template problem. Major template errors are often a sign of a weaker description, but certainly aren't always. Description is always going to be my first focus. If you beat the other item in terms of what the item DOES, then you win my vote. Decently done Auras, CL, and prices are next. Template? I mean, it's important... Now, if we're talking two Superstar items in a face-off... well, then I'm still going to go through pretty much the same order. Description > Aura, CL, Price > Template. There's always a lot of talk about "well, a good freelancer would..." and I can't really speak to that. But it seems to me that when an editor returns a submission and says "Stop screwing up this formatting," then you figure it out pretty quick. But when an editor replies with "Stop sending me boring items," I'm not really sure what the next step is. I'd like to see the good ideas first.
Because somewhere in the vast web of Paizo's tech spiders, there is a Real Name next to any item. And if they happen to see your false posts, someone is going to investigate, and they won't see the right name. That submission is not DQed. However, now I, as a voter, think that false name is attached to the item, so my votes are still skewed. That submission is potentially in lots of trouble. These are not in any way the same thing. But both of them suck a lot.
Because the end result is the same. As a voter, I know that Jack wrote that +1 sword of Doom. So instead of me voting for the sword, I'm voting for Jack. Sorry, Jack, but that means you're out. I advised every single person I know at our local PFS not to reveal their items to people outside their workshop groups. The odds of seeing a particular item are dismal, so the benefits will never never never outweigh the penalties of some random person posting online and linking my item to my name.
I've read every single post, and that's exactly how I feel. On the other hand, at least they're talking about my item? There's no such thing as bad press? Ugh.
Here's a quick 10-days-out reminder that the next round is shorter than you think it is. Make a plan now. If your item makes it though the next cull, reward yourself! By drawing a dang map. Do the same with the next cull... and the next...
Good ideas are a dime a dozen. Fully-realized, well-edited, concrete realizations of them are very rare.Totally disagree. To be honest, the idea behind a module is not that important, much like the plot of a kung fu movie is not that important. It just needs to be strong enough to do its job and enable adventuring. See, I know folks are going to disagree on this. One of the biggest things I've learned in this competition is that we all have different standards about what we think a "superstar" idea is. It actually terrifies me at the start of the season, and it's one of my biggest pet peeves about the snark thread. We all have different ideas about what should be snarked and what should be supported. I love a good investigative module, but I also love a solid delve with challenging encounters. It just depends on my mood. And those two module types require vastly different skill sets in the author. I don't know many people who can do them both. I am always terrified that all of the voters aren't in the same mood I'm in.
Just had my first BIG DECISION moment. On the left was a great idea with a cool theme that just needed to sharpen up the mechanics and abilities. On the right was a good idea with well developed crunch, though the theme needed a little more work. Both were formatted properly, though I decided not to focus on nitpicking template issues as the tiebreaker to make this decision. In the end, I went with the option on the right. I have to show some more respect to solidly crafted mechanics in this competition. But someday, I'm going to write a character that needs that item on the left, so I want to see some rewrites soon! So yeah, if you're the kind of person who votes concepts over crunch, don't worry, I'm here to skew the algorithm the other way! :)
Good news: On route to Dedicated! Bad news: Far too many projects this week plus PAX, so no Marathon for me. Worse news: Still haven't seen my item, though I know others have. But I need to KNOW.
Aha! I have finally seen snark (somewhere within the last 1,460 posts!) which I truly believe is mine... though it sheds some light on a few other snark contenders as well... HEY HEY I GOTS SNARK!
General snark: I don't like how middling items keep recurring for me and are consistently paired up with really poor items. I'm voting for the middling items so much (sometimes as much as 4/day) that I think it's inflating their actual worth. Don't worry, that isn't how the algorithm works. It creates a long list of rankings based on how each entry relates to the others. All you've done is say that the "middling" items are better than the "really poor" items. You haven't ever said the "middling" items are better than the "really awesome" items, so they won't be ranked higher for it.
Even though it isn't appropriate for the competition, I always love reading the backstories that have been added to some of these items. I like that authors care enough about their item to place them into the lore of the world, whether or not that's Golarion. It's my most favoritest dealbreaker.
Thanks for the laugh, joke item. I'm still waiting for my first jaw-droppingly amazing item, so you currently win the Most Memorable After Ten Votes award. Congratulations?
I actually thought the submission deadline was on Friday at 2pm, so when I went to see if I could start voting yet and realized my mistake... well now I'm pretty much completely calm and in waiting mode. So, my advice is to fail a Knowledge: Rules check (DC 5), and then confuse yourself into a state of steadiness. It's basically like Drunken Boxing. I think. On the other hand, I finally closed the tab with my submission so I'd stop reading it. Hopefully the next time I read my submission text will be when I get to vote for it! I've got zero time for hindsight this weekend!
I'm a little overwhelmed that everything starts at the same time as the start of the school year, but at least the first few months of calculus and physics are pretty easy to teach, right? I can't wait to start seeing items once voting opens! I just may have to skip hitting Marathon this year. |
