I should have looked that one up. Yes I'm old school and still think about infravision and ultravision - but it says clearly in the current rules "black and white".
Our party elf has darkvision, as do all of the other party members - but not so for other elf allies, who probably wouldn't remain still in an area that was totally dark (they prefer low-light and would find that unnatural, plus they don't sleep).
In an area covered in darkness,
Would there be any discernable difference between drow and elves?
I think in conditions of light or low light coloration would be obvious, but if that is the only distinction then darkvision would not reveal the race. The creatures are the same type and subtype (humanoid, elf).
Of course, if you have only 60' darkvision and are pursuing drow who have 120' range, they may see you first and leave or take cover (unless you are very stealthy).
This may belong on the rules forum, but I'll start here.
Presuming identical gear and no extended observation, using darkvision in the dark, is it possible to distinguish drow from elves?
Or are their shapes and features substantially the same, other than color?
I ask because we are doing the siege and counterattacks in a campaign mode style of play, and we have over 600 elves in the surrounding woods, not including the drow that venture forth.
Granted we will normally have dim light outdoors when folks get into nighttime visual ranges, but without knowing exactly where everyone else would be, surprise attacks in darkness could be a bad choice for hasty attackers.
My PCs all have darkvision, and may be as likely to attack in the dark as the drow - although I will encourage defending parts of the elven encampment at night when the PC's allies don't have advantages to attack range and vision.
I'm organizing PFS and SFS sessions at a convention.
The session tracking sheets on the Paizo site only show PFS stuff - no SFS factions and such.
Is there a version hiding somewhere that has been modified for use with SFS?
Or is the best thing to print out a full tracking sheet from each scenario, quest or special?
After getting beat up by a wight whose behavior was written as "not to pursue", the PCs retreated, left the dungeon and rested overnight - coming back healed up and refreshed.
There is no evil cleric in the dungeon to hit the wight with negative energy.
Does the creature also regain hit points overnight? Have you seen a rule on this anywhere? In the absence of anything to the contrary, I think it should regain hit points just like a living creature. It is not a construct, and I would think the force that brought it to unlife would slowly creep back in unless it is destroyed.
I'm thinking about putting them on a werewolf and it feels a bit cheesy to have them meld into wolf form and gain the benefits of quadruped movement speed and an enhancement from a single pair of boots.
For the character concept and the current campaign setting, these boots sound like s very good deal. If we stay outdoors on open plains with deep snow, it seems like I could move 200' in a round when most PCs and opponents might be limited to 15' with a double move.
What is intended by "normal speed" in the descrition for this item? Does it remove all speed penalties for moving over snow and ice, or does it just eliminate tracks and falling chance at the "normal" (reduced) speeds?
Also, if the terrain is otherwise open (except for ice or snow of any depth), does it mean you can use running movement rates? Do these boots remove only the movement penalties or do they actually make it so these surfaces are no longer difficult terrain for the wearer?
With wizards is it often useful to buy a scroll of a spell that is a level higher than they can't currently cast, and use it if needed, risking only a small spell failure chance.
But alchemists can't use spell completion items like scrolls, and I thought I read that wands for "spells" that only appear on alchemist's extract list don't exist because they can't craft them without a caster level and wizards don't have access to spells of the same name.
I'm not sure if brew potion has a similar limitation, or if potions and wands both exist via the assistance of master craftsman.
What I'm really looking to determine is if there is any way that a 5th level alchemist (who normally only gets extracts up to 2nd level) can obtain a 3rd level extract, and somehow use spell slots and the infusion discovery to make infusions for other party members.
Basically is there any way to supply such items to the rest of the party for an average cost that is less than buying potions.
The spell in question is on the wizard list, so wands and potions are possible, but I want a use-activated item for non-casters and I'd like to know if there is any more economical way than potions to accomplish multiple castings.
AFAIK there is no way to do this. Items that increase your caster level exist, but I don't recall any that increase your class level or open up higher level slots. Metamagic rods could get you a higher level spell but I don't think those work for alchemists and even if they did they don't give access to spells with higher base level (only modify level of spells you can already access).
Your math for a cylinder checks out. Figuring 15' reach plus a foot to wrap around an ankle for trip attacks, diameter comes out to 5/8" and thus about 3 hp.
A real whip is more like a section of a long cone - thicker near the handle and thinner at the supersonic tip. People are more likely to attempt to slash it near the tip, where the diameter would be thinner, and without trying to get too precise about the diameter at the tip or the position being attacked, it might be reasonable to presume that somewhere within a few feet of the business end the diameter is only 3/8" or 2 hp.
A metal whip will have greater hardness and hit points per inch, but would be also at least 8 times as dense and only 2.5 times the weight, so you could do the same work with a significantly thinner weapon - nearly a factor of 2 reduction in diameter.
With steel at 30 hp/inch and 5/16" or 3/16" diameter, you'd be talking about 10 hp or 6 hp for a 5 lb stinging whip. I would tend to use the lower number, but if you are that detailed keep in mind that breaking off the last 5' of a whip still leaves you with something that can reach 10'. It would be a houserule if you wanted to shorten the whip rather than make it completely useless.
To John and the development team:
Do you expect each of these new scenarios can be completed in a regular convention slot of 5 hours maximum duration?
My experience so far with higher level adventures is that it takes more time for players and GMs to figure out what to do with the many options accumulated at those levels.
I am excited about this and thinking about offering the arc at a convention this fall - but I want to make sure we slot enough time to do it right.
Any guidance on that?
I'm thinking about creating a scenario where people play Scooby and the gang using PFRPG rules and pre-generated characters.
I want Scooby to be an awakened animal and a full-fledged PC, and the other characters to have abilities and flaws similar to the rest of the gang.
All the humans should have at least one level of investigator, with appropriate archetypes. PC level probably about 3rd so dips into other classes are possible and there are some interesting feats and class features.
Characters will have almost no magic or spells or weapons or armor, but typical wealth and mundane, alchemical, occult and tech items that fit the iconic characters. They should also have spent a good portion of wealth on items like books, trap making equipment, a vehicle and stashes of food.
I'm looking for suggestions for character builds, if any of you are inclined to share your ideas. My goal is to match iconic character abilities, skills, equipment and behavior as much as possible for roleplaying - with a group that relies mostly on their wits and luck to solve mysteries and doesn't have the typical capabilities of a party that regularly engages in combat.
Once I've settled on the character builds, I'll develop a familiar story with each act incorporating a classic opponent of appropriate CR.
RAW states that: "A wizard also selects a number of additional 1st-level spells equal to his Intelligence modifier to add to the spell book".
From this sentence alone, it seems clear that
But the sentence uses the words "also" and "additional" which refer you back to the previous sentence, which starts with "A wizard begins play with". This language is somewhat unique and in context it could be interpreted to mean "At first level a wizard gets a spell book with a number of 1st-level spells equal to 3 plus his INT modifier at the time he takes his first wizard level."
The question is whether the unique wording suggests something unique in the RAI or it is just leftover language that wasn't cleaned up. The 3.5 language isn't significantly different, and IMO the intent was to represent a pre-adventuring lifetime of study where a wizard with high INT could leverage it to get a slight advantage at first level.
I think the hang up with "all means all" here (I'm not the only one) is that spells in a spell book represent something physical and tangible that a character would otherwise have to pay and spend time to have (and might seem difficult to abstract away if the bonus goes away). But if you get free spells when levels increase then why not also when INT increases?
The "begins play" in the first sentence leads one to believe this is a something unique that you only get at 1st-level. If you interpret that the language simply didn't consider later INT increases, you could argue that "1st level spells" in the second sentence should be "spells of the highest level you can currently cast, or lower".
I guess for simplicity's sake I'll consider the list of spells in the book to be an abstraction which moves up or down with INT and/or level. And to match RAW the extra spells from INT will be 1st-level, even though they weren't much to gain at levels 8-10 when INT modifier increased. That will net the character 3 more spells or the equivalent of 45 gp in borrowed spell books and inks it would otherwise cost for those spells.
I am not adding the INT increase-derived free spells unless I receive confirmation that Hero Lab follows an official ruling in this case.
One thought against adding spells due to INT increase is that the increase already affects your chances to learn new spells. But at this point spellcraft modifier is such that checks to learn are already automatic.
I was just using Hero Lab this evening and happened upon this new feature. My wizard started with 18 INT and now has 24 due to 2 points of increase from level advancement and 4 points from a headband. Hero Lab indicates a free spell total that corresponds to the higher INT number. It also does not seem to track whether your free spells of any given level exceed the maximum number of free spells you can have for that spell level.
I know I should ask LWD about this, but I'm wondering if they got some new official rule on this from Paizo. Furthermore there is a question of whether or not extra spell book spells from increased INT need to be 1st level spells or could be of any level the PC can use at that time. The 3 spells you "begin play" with when you have your first wizard level are clearly 1st level, but the part about more spells in books due to high INT is a completely different sentence. The cost of 3 first level spells is not a big deal, but if the PC can have 3 more spells of 4th or 5th level, the cost savings is no longer trivial. Generally INT is not going to increase until you are already capable of casting spells higher than 1st level.
Note that this is not just a temporary bonus question, as some of the INT increase is due to advancing in character levels - which is permanent unless you are unfortunate enough to get permanent negative levels. If you choose to get smarter, learning an additional free spell of a level you can cast would not be an unreasonable reward. It really isn't a huge deal for a PC that between free and purchased spells has well over 100 spells in her books. If you get any free spell at higher levels, it seems like it should be something of value to your character. The two free spells per level and the optional favored class bonus spells track with your current level (highest spell level and one level lower respective maximum levels). I think the same should follow if you are spending ability score increases or item purchases on INT. By the time you can afford that, you may have learned for free or purchased all the 1st level spells you care to have.
One more clarification:
If you want to gift the items, do you need to click the gift link on the Humble site, or can you do that when you take the codes to the Paizo site?
My thinking is that I want to get the certificates into my Paizo account ASAP and assign them later from there. But I'm not sure if one option or the other makes it easier for me to present the gift. As I don't have a humble account I guess Paizo would help me keep track of where these get distributed.
Thanks for the quick reply and the source, TOZ!
And thanks to Match_stick for the clarification.
Here comes Santa Claus...
I'm not sure if this would help traffic but I'm thinking about paying for bundles now and distributing codes later as convention prizes to be redeemed a few months from now.
Can that be done, or do the bundles need to be redeemed within the promo period?
Also, do you get a single code with each purchase, or is it broken up into different codes for the items included at each different award level?
Diego, nice catch on the confusion mechanic.
I thought it seemed wrong that they would be immediately confused and lose their actions, and you found the only place that clearly confirmed they would get to complete an action before the full effect of the confusion set in.
That makes it a little more survivable, and more or less matches how it played out. The PCs killed one in the first round while the other two kept making two party members try to kill each other, and the rest of the group had to split actions between protecting their friends from each other and actually going after the monsters.
Is there a rules source that explains this?
Seems to be game breaking if you have a circumstance where you have to succeed on multiple saves in the middle of your action (or lose it) due to an effect that doesn't require line of sight and could be concealed to prevent detection.
The chance of succeeding on multiple saves gets exponentially more difficult with more creatures, so the encounter CR should not go up simply in proportion to the number of creatures. Especially if the aura radius is beyond your range to perceive and identify the threat.
As an example I refer to 3 or more Seugathi all concealed in a fog filled room with auras that extend beyond the only entrance to the room. Unless the party is prepared with a way to clear the fog from a distance or blasts spells blindly into the room, this is a likely TPK unless the party numbers several greater than the monsters. The chance of succeeding at 3 DC 20 will saves at the same time is minuscule for most PCs, so while the monsters control 3 party members and remaining party members have only a 1/4 chance of acting normally, the party almost surely destroys itself or goes insane before it can take out the opponents. The only good chance for the party is if the GM decides the creatures start with aura down and must get to their initiative to use a free action to activate it. They need a chance to take out one of the creatures early because 3 aura saves stacked together is just too powerful if it catches them by surprise. Probably a dumb move to enter the room here, and PCs might not notice immediately if a fellow party member is confused. Of course the group was bold about the lack of good visibility as they did have a telepathic bond, which in hindsight might have had some weird effects or chance to identify the cause when others began to get confused.
As a follow up for that creature, text says it can suppress or activate aura as a free action.
That might suggest that it has to take a turn before it can have its aura active, but it could be just as likely that it normally has it active and needs its first turn before it can deactivate it.
Which brings me back to the original question - what happens if the aura is up when you charge into it?
Say a PC sees a bunch of baddies and decides to charge in and slash at one with his sword.
If during the move to close he encounters the perimeter of an aura, does he have to stop there and make a saving throw, potentially ending his move action and denying his attack action?
It seems to me the aura should only take affect once you start your turn within its boundaries. Most say save each round which suggests you need to be in there for the better part of a round before it can affect you.
If it can immediately interrupt turns I think closely spaced groups of creatures with auras could be easily overpowering; multiple chances to fail before you get a chance to do anything to reduce opponent numbers.
Yes colony of Seugathi I am looking at you - if I can even see you before I stumble into the range of your auras.
@ GM Lamplighter: Thanks for that reminder. I am giving this careful consideration as the space is limited.
We only have to contend with the noise from a small number of our own tables, but the space is only 700 sq ft. I think we can squeeze in 4 or 5 small round tables without noise being a big issue, but 6 would put too many too close to each other IMO.
Ideally I like to have all participants at a given table be closer to each other than they are to anyone at a neighboring table.
Thanks for all of the comments!
If others have experience running these, please continue to reply with run length info.
The only module on the list which I saw advertised as a 64 pager was Tears at Bitter Manor, which for PFS shows two separate chronicles so I was already figuring it could be 5 or 6 sessions to run the pair.
We are not completely restricted on time (private space and 24 hour rental rate) but we may be limited by player and GM endurance. I read a review that indicated folks started to tire of the Harrowing after a couple dozen cards and figured folks probably found a way to wrap before all 54 cards came into play.
Part of the draw for our events is that we do dare to try things not done at other conventions, which our players haven't yet had an opportunity to play. Many players seem to like scenario story arcs, modules and even AP sections. So I want to put a few offerings out there which can completely fill an entire day with exciting action. And we usually have super GMs who are willing to make that happen if we provide enough time for them to do it right.
I am also planning to put an open gaming slot at the end of the event in case someone really needs the overflow time, but a one-day marathon event is often easier for people to commit to than returning to finish on another day. If most people say the high level mods will take at least 4 slots, I may have to rethink those as possible choices, but they sound absolutely awesome if we can fit them in.
I am planning to offer some modules again at an upcoming convention.
We have no problem triple-slotting modules that need it, but would some of these typically finish in less time and be better suited for 2 slots in the schedule?
We will have 5 hour slots or 4.5 with breaks.
How many hours have you seen the following modules take to play?
1-2 The Godsmouth Heresy
Do you have a rules source that defines "perfectly smooth" this way? It is a big deal to increase the DC from 20-30 to infinite. Realistically, if the surface imperfections just need to be significant in size compared to the grabbing appendages. Real geckos can move on a typical "flat" ceiling. The bumps need to be larger for an 11' long version, but rafters or stone blocks would do. Also, the bigger the room, the less likely that the ceiling will be a single surface with no exposed beams.
Thanks. I missed the bold bit about it being the same opportunity, which makes sense. The straight line limtation, if imposed, limits the usefulness of the tactic significantly if you want to position for a party flank. I will look at the other maneuvers to see what they imply about unintentional movement. Footnote for feather fall suggests falling provokes if it moves you out of a threatened space.
Actually the two ki powers replace slow fall 30 and high jump, so would have to ditch one of those to take elemental fury, or give up another feat to get elemental fist (which isn't really worth it without the archetype).
Also I missed that you now need to obtain ki powers for sudden speed and furious defense before you can use ki points for movement or AC (unlike the standard monk). Sudden speed is +30 instead of +20 and lasts a minute instead of a round. The monk has to be 7th level before he can take furious defense, although formless mastery appears to be a better choice if the monk doesn't have a style feat (of course the current version does).
But he would also get to add a style strike.
We will have some things to talk about...
My son has a monk character and we are considering a rebuild. The original is a Monk of the 4 Winds archetype, which won't be allowed with the rebuild. He would still take elemental fist as a feat, but it looks like there is no way to get the damage progression (or extra uses) on the elemental fist attack once you lose the archetype?
Is there another way to boost that ability that is unchained legal?
(edit) I guess elemental fury ki power is the obvious replacement. Doesn't allow the damage to get an extra 1d6 per use at 5th level, but for a ki point could add (at 6th level) 1d6 to every attack for a 3 round period. That could be 9 attacks with flurry or 12 if he spent 3 more ki points (or more attacks adding AoO).
So instead of 2d6 six times per day with elemental fist feat he gets 1d6 for all attacks in 3 rounds whenever he spends 1 ki point, but he gets back stunning fist which he had given up for the archetype. Have to expend 2 ki to get similar elemental damage in a day.
And of course he gets better BAB, no flurry attack penalty, and a couple of ki powers instead of slow fall 30. Slow fall any distance is now available as a ki power choice, so he gets a bonus to falling distance as well as another power. The only other downside is reduced will saves.
Cao Phen wrote:
Do you have an estimate, for PFS mode, of how long it would take to play the sanctioned content in Plunder & Peril? e.g. could each of the 3 parts be played 5 hours or 6 or 8 or 10? For a 3 day event, we probably have at most 35 hours of gaming (7 slots of 5 hours each).
If we wanted PCs to start at lower levels (playing some scenarios and then part 1 of the Skull & Shackles AP first), I'm sure we could NOT get though all of Plunder & Peril at the same event. Although maybe it would be better to just open it to PCs of the requisite level and run it campaign mode with one level per day. The other option would be to build PCs up from level 1 during this weekend, and try to have them ready to play Plunder & Peril at a future event.
It's more of a gimmick idea, to not have a "balanced" party, but instead to have too many pirate leaders, each trying to show up the rest with bravado and derring-do. What happens if everyone wants to be the pirate captain, and nobody wants to have a different role in the crew? Perhaps one stands out and becomes the pirate king, or maybe it just turns into general silliness and mayhem with a crew that knows swashbuckling and nothing else.
If I did this with a higher level game, I think I'd relax the restriction to allow some amount of multi-classing; it might too severely limit the chance of success in some scenarios if the party lacks diversity.
I'm considering having a pirate themed event, and possibly a session that is only open to swashbuckler PCs.
Any ideas on what might be some of the best choices for the latter?
I've seen quite a few PFS scenarios with pirates or ships, but I suspect there are several other good options out there which I've yet to consider.
I found the answer after searching some other posts - it's in the combat section of the PRD:
Supernatural Abilities (Su): Using a supernatural ability is usually a standard action (unless defined otherwise by the ability's description). Its use cannot be disrupted, does not require concentration, and does not provoke attacks of opportunity.
and for rage:
barbarian class wrote:
While in rage, a barbarian cannot use any Charisma-, Dexterity-, or Intelligence-based skills (except Acrobatics, Fly, Intimidate, and Ride) or any ability that requires patience or concentration.
So the supernatural ability doesn't require concentration and it's not a skill. As long as the GM rules that it doesn't require patience (can't see that this one would) - I should be good to go.
I'm looking at a build taking a 1-level dip in barbarian (savage technologist) and a wizard with air elemental school.
The initial thought was to get the fast movement and extra HP and to not suffer AC penalties if the character runs out of spells and has to rage.
But can the supernatural ability Lightning Flash be used while raging?
I'm imagining the character charging into the nearest crowd of baddies and explosively discharging electricity all around.
If this can't be done, I'm a bit at a loss to come up with the build. I had not really planned for any great skill with a weapon - thinking of something more like Marvel's Storm, unleashing wind and electricity when she loses her temper.
The PC I'm considering actually already has a level of Cavalier, but it will take a while to get 5 ranks in perform (which are required for Ensemble).
The character also has some prestige, which I'm willing to spend on the vanities. I'm looking more for something I can do right now, versus waiting several levels to build a certain game mechanic. It isn't so much about having bonuses to bardic performance as it is having followers that are PFS legal and match the concept.
If there's something out there of this type that also grants a little bonus, then that would be, well, a bonus.
Ensemble might be interesting when he gets high enough level to take it. The downside on harmonizing armor sounds a bit scary.
What I'm really looking for is whether there are sources for vanities other than the Field Guide. The followers in the guide aren't exactly a match for what I'm envisioning, nor are the businesses. Theater and Concert Hall are close for the latter, but I'd rather have a business that's a roving band.
I could (for no mechanical game effect) have followers that are heralds, porters, seneschals, etc. and just say they can sing and play instruments I summon for them, but I'd rather check first to make sure there isn't a better fit in another resource.
Geoffrey Griffith wrote:
Try to recruit party members. They can make perform (sing) checks untrained. I have a friend who's bard is wildly popular. He has the flagbearer feat, and carries the flag of his band, and offers party members the opportunity to be roadies and what not and everyone seems to enjoy the chance to participate.
I like this from a role play aspect, but I don't think there's really a game mechanic for it. If the other PCs don't have a level in bard, they can't aid a bardic performance. But they could aid to a general perform (sing) check to get the crowd's attention. But in organized play I think the general performance can only be used once per session as a day job check, and you can't get help with those. Where is this flagbearer feat? Do you mean cavalier banner?
For my next character, I plan on making a Bard, and I'd like him to have a troupe of back up singers. Is there a vanity that would best fit this concept? I suppose I could have them be a bunch of heralds, but am wondering if there is something more specific.
I also plan to have a signature song for each spell or ability that requires perform (sing), and have an audio clip on my phone that I can play when it is his turn to act. With the small number of spells known, I might need to cycle through a couple of song choices per spell or ability to keep this fresh with repeated use.
The initial focus for spells known will be enchantment/charm spells, and I already have a few clips teed up. I'm sure I'll get a few other spell/song suggestions in response to this post. If I use your idea, I'll try to give you credit in game.
We are thinking about adding one weekend a month to our local play schedule, focusing on the Emerald Spire.
I'd like to know what folks are seeing for typical run time for each of the levels. Can you make it through the sanctioned content in most levels in 4-6 hours, or should we plan on extended or split sessions?
Of course, I'd like to complete levels without having to split them up over multiple sessions (and all the keeping track of people who attend one session but not the next). But if levels run much longer than 4-6 hours it would require me to make special arrangements with the hosting gaming stores or move the games to private venues.
Has anyone seen anything recently about upcoming holiday boons?
I've happened upon a few of these in the past and am wondering if anything might be planned that coincides with an upcoming convention.
I did find this wiki of Golarion holidays, but they obviously all don't result in boons.
What I'd really like to see is some temporary boons that only apply during the feast day or festival week, if the occasion is celebrated in the region where you are adventuring (or, if a religious holiday, if you follow the corresponding deity). Granted that would limit when and where or by whom the boons could be used, but the upside is they could all be created one time and re-used year after year only when, where or with whom they apply.
Of course, the expiration idea might require sort of the opposite of the current boon usage - you'd have to receive the boon as you are seated at the table, because your next session might not be until the festival is over. Or it could just be a convention-only thing: after you play one session you receive a boon that just lasts until the end of the holiday or for the weekend or for one week.
Would the flatter distributions be attributable to being more prone to influence by the individual GM than the scenario itself?
Possibly. The theory is that the table bias (due to GM, PC mix or player choices) should average out as more tables are included in the set of reviews. But the data sets are generally so small that I don't think you can say with much confidence that the scenario is prone to GM variation. More likely there were just a few people who had differing experiences (more due to the people than the scenario).
If you truly had two different ways the scenario might be run, which resulted in different levels of enjoyment, with enough reviews you might actually get a clearly bimodal distribution of ratings.
You actually can (and I did) calculate something called a bimodality coefficient for each distribution, and based on the result you can see if the distribution tends toward bimodal, unimodal, or uniform (equal values at all rating numbers). The flatter distributions most closely resemble a uniform distribution, which has no clear "typical" value(s) regardless of the result you'd get by calculating an average rating.
If there are a lot of responses and the distribution of ratings is still very uniform, then YMMV: your likely experience (and rating) with such a scenario might be nearly random. The scenarios that do have many reviews tend to NOT be like this; they tend to have a clear peak in the distribution of ratings and very few ratings far from that value.
Only a few scenarios have a highly bimodal distribution of reviews and more than a handful of reviews at the secondary peak: Library of the Lion, Murder on the Throaty Mermaid, The Elven Entanglement and The Waking Rune. It may be that these can go really well with the right PC mix and really poorly with the wrong mix. Other scenarios may be better balanced to work with a larger range of PC parties and capabilities.
Another way to look at it - the average isn't a great measure of typical value if there aren't a large percentage of responses near the average.
Many of the distributions of review ratings don't have a single clear mode (or two clear modes if responses are split). These distributions more closely approximate a uniform distribution than a unimodal, bimodal or normal distribution.
With more reviews, the most common ratings are likely to emerge, with counts at the peaks that exceed the counts for other rating values by more than just a couple or a handful.
10 reviews was a good cutoff to make sure the report could include a majority of the scenarios, but unless they are all tightly grouped you probably need more like 20 before the distribution of responses has a fairly clear shape. Unfortunately we currently only have a few scenarios that have received 20 or more reviews.
DM Beckett wrote:
Two things to consider, is that he did specify scenarios with at least 10 reviews, and also that those numbers might have changed as people here have said "hey, I liked that scenario, but it's rated low, let me change that." or "wow, people didn't like that one, but that's my favorite, can't let that stand."
Understood. My point is that there is considerably higher confidence in the average for a scenario that has 30 reviews compared to one with only 10 reviews, and some meaning can be gleaned from sites with even fewer than 10 reviews, e.g. one that received 7 reviews and they were all 5 stars.
In terms of keeping up with the new data that comes in, that would be a daunting task, especially if it's done by scrolling through the reviews on the web pages to pick off the numbers for each one (both for changes and for new reviews). I'm not sure how often Kyle may try to update this, I'd guess not more than weekly.
It is also far from an unbiased system, as you point out. Everyone can look at all the other responses, and there is a tendency to not respond if you agree with the general consensus but submit a conflicting review if you disagree. That tends to flatten the distribution of responses (more near the extremes) and reduce the count of reviews for scenarios where there is close agreement on the rating.
There is still a lot of good data collected here, but folks should not be misled by averages reported to a precision of 0.01 stars. In many cases (even with 10 or more reviews) there isn't a lot of confidence that a more thorough polling would result in average ratings within half a star or even a whole star or more from the current values.
Of course, with more reviews, the confidence in the average ratings improves. But it is still just a one number rating system, so there is only so far you can go to improve the confidence without taking steps to eliminate bias and to separate the different aspects that go into creating a high or low rating on this scale.
I found some minor errors in the season averages (the last scenario was accidentally left out of the calculations for seasons 0, 1, 2 and 4).
I also added some double-check calculations and some further statistical stuff to the "All Scenarios" tab, such as response range, standard deviation, skewness, excess kurtosis and 95% confidence intervals for what the true average might be if we had more reviews but the same standard deviation for the distribution.
This last item suggests a slightly different ranking, taking into account that the average for a scenario with a smaller number of reviews is less likely to represent what we'd get if more people submitted reviews, but a tighter grouping of ratings suggests a higher degree of confidence even if there are few reviews. Many reviews combined with a tight grouping suggests the highest degree of confidence that the results are reliable or repeatable.
Unfortunately I'm still a newbie to Google Drive, and I didn't want to modify Kyle's original or overwrite a shared file while others may be accessing it. I've downloaded it and made changes offline and will send to Kyle to figure out if and how he may want to incorporate it.
There really is no absolute best ranking with the small amount of review ratings we have for some scenarios, but with filter and sort controls in the Excel version it is easy to generate a variety of top 10 or top 25 lists (most skewed, most split, most consistent, etc.)
In the season summary, season 3 had the most reviews, the highest average, the most skew towards a majority of high ratings, and the closest to a normal distribution of results. Distributions of ratings for other seasons were flatter, with averages closer to 3.
I don't know. I presume they chose to code it that way on purpose in case someone wanted to comment but not provide a star rating. It appears that the average ratings on the website exclude those responses from the numerical averages, but the comments are still presented for others to peruse. It is a small percentage in most cases.