|Gark the Goblin|
|6 people marked this as a favorite.|
In this thread, I am hoping to gather statistical evidence of the martial-caster disparity, or of the lack thereof. Since it is a very hot-button issue on these forums, I want to ask everyone to please be chill and not like flame or whatever. The idea is just to see if we can add new evidence (beyond play experience, which is still likely the most valid) to the discussion.
Also, I should state some caveats: I am a pretty strong believer in the disparity, and I really only know enough statistics to be dangerous (some college classes). I couldn't find a test appropriate to this data, but did a Z test for difference between proportions to at least show off some of the data I collected. As I have not seen any similar efforts on the boards here, I figure questionable results are better than none, and will hopefully spur brighter minds to make their own comparisons.
The analyses I conducted checked to see if any given class or race was significantly more prone to death. Since I have only fully read through the Rise of the Runelords and Serpent's Skull APs, I chose those subforums and collected data from 1) obituary threads and 2) threads where players and GMs posted their starting parties' compositions. The second set of data established a baseline - if, say, 1 out of 10 characters created in Serpent's Skull is a ranger, then if deaths were unbiased we would expect 1 out of 10 deaths to be of a ranger - but as you might expect, the first set of data was often quite different from this baseline! A statistical approach was necessary to determine if deaths were "biased" or not.
I calculated Z scores for each* class and race to test whether or not two proportions were different. The two proportions were 1) the number of character deaths of class/race** A divided by the total number of character deaths and 2) the number of characters of class/race A recorded as being created divided by the total number of characters created. Basically, I was testing to see if the proportion of deaths was similar to the proportion of characters of class/race A created - if they were similar, it meant that race or class was having its "fair share" of deaths. A Z test calculates a "Z score," and you can either use this to calculate the probability that the proportions are truly different (inconvenient; requires referencing a table for each class and race), or you can choose a probability cutoff beyond which you say the differences in proportion are not solely the result of random noise. I chose the second option and set the probability cutoff at 0.05. Since I wanted to check whether a class/race was significantly more death-prone AND whether it was significantly less death-prone, I wanted a two-tailed test, which set my Z score cutoffs at -1.96 and +1.96.
*Classes and races which did not have at least one death entry AND one character created entry were ignored. Technically, I should have ignored every one that didn't have at least 10 entries, but then the results would have been very trivial.
**For the Serpent's Skull thread I assumed that anyone not listed as a specific race was human, but when I applied this method to the Rise of the Runelords thread I didn't realize there was no race suggestion in the obituary template. So, "human" deaths are likely over-reported in RotR.
The links below are graphs! Note that each individual class and race in each AP had its own individual two-tailed Z test for difference of proportions. A positive value means my data have that class/race dying more often than one would expect, and a negative value means my data have this class/race dying less often than one would expect - but this does NOT NECESSARILY mean the differences in these data are statistically significant. The difference is only significant if a bar tops 1.96 (spoilers: none of them do).
Z scores for Rise of the Runelords classes; n1=52 and n2=125
Graph for RotR races omitted due to aforementioned issue with data potentially over-reporting human deaths.
Z scores for Serpent's Skull classes; n1=47 and n2=170
Z scores for Serpent's Skull races; n1=41 and n2=173
"Bad" here only means that there are significantly more deaths reported than there should be if death occurred equally often to all classes and species.
There is no significant difference.
With the major assumption that this Z test is a valid approach (and it's not), there was no significant difference between the proportion of characters (of class/race A) reported as dead and the proportion of characters (of class/race A) reported as being created. This means that I found no significant evidence that fighters are dying more than anyone else, although from some of the graphs it certainly looks that way. It also means I found no significant evidence that rogues die less than other classes, or that wizards die more.
While writing this up, I thought of a better test: divide up classes into caster and non-caster, which will increase sample sizes drastically. I'll try that tomorrow, perhaps after collecting another few pages of data. Also on my to-do list is to check if non-core classes and races are more or less likely to die than core.
I also want to note that character deaths are probably not the best measure of fun. That ranger who keeps dying keeps getting brought back, because he puts himself on the front line and has fun shielding his allies. Surveys and any of the other, more robust statistical methods out there would be valuable to the discussion, I think.
Rise of the Runelords parties (page 9 only) - recorded number of PCs of each class and race
Rise of the Runelords Obituaries (page 22 only) - recorded number of deaths of each class and race
Serpent's Skull parties (page 2 only) - recorded number of PCs of each class and race
Serpent's Skull Obituaries (pages 4 and 5) - recorded number of deaths of each class and race
Raw data is here
Sampling the most recent pages of some threads is sufficiently random.
Unchanging populations (i.e. even though I was grabbing SS party composition posts from 2011 AND from 2016, the average party composition has not significantly changed). A big assumption, given that people are using new books and new classes and new races!
Sample sizes are large enough - MAJOR ASSUMPTION. Populations are likely much larger than the samples, but numbers of "successes" (deaths, or characters being created) were only above 10 for humans and below 10 for everything else.
Null hypothesis: p1=p2. Significance level: 0.05.
Let me know if you spot any errors! It's been over a year since I took these classes and I'm working off some pretty fuzzy memories.