|
Greetings Mortals--
So the Pathfinder Society Community Survey is now closed. I appreciate everyone's responses and everyone who pushed the survey to their local gaming groups.
I should have the data available for Community download shortly. At this time, I believe I will be able to put it into a google document with open access.
As usual, I respect thoughtful and constructive comments and suggestions on this survey. I have never claimed to be a wizard with school focus: marketing or two-hand wielding keyboard fighter with the survey archetype. At best, I'm a level two goon with the Dilettante feat and a few ranks of making-stuff-up and not-being-afraid-to-fail. As a philosophy for this survey, my motto was: "Do not let the perfect be the enemy of the good." At a certain point, I had to say "Piddlespot it!" in regards to the survey and just go with it.
I avoided discussing the survey in detail while it was still open to keep focus on the survey itself. I'm going to open with answering a few posts that I sadly ignored, but feel is important to acknowledge:
One thing that really annoyed me is that, like a few people mentioned, there are a lot of times either none of the answers really applied, but no answer wasnt the right way either, or that i really flt I should clarify something. Can I suggest that all questions have a way to type something in?
I am both deployed, and play primarily PFS PbP, so that influenced a lot of my answers, but there wasn't a way to mention that.
One of the things that I struggled with is how to balance length with getting the depth and breadth of topics that I wanted information on. If you've read my other posts of note, you know that I'm not shy about talking way too much.
But with a survey, it's a different animal altogether: if I had been able to ask all the questions I wanted with all the detail I wanted, the survey might have been 300 questions and taken 2 hours to complete.
Beckett, you make a good point. I chose to exclude, perhaps unfairly, PbP play from this survey. There just wasn't room.
As far as having more spots for open-ended questions, that too, leads to problems. The strength of a survey is getting useful numbers rather than dialogue and then being able to do some statistical analysis on that. Too much open-ended and all we really have is a bunch of stuff that doesn't tell us much statistically.
There were a few editing glitches, and I found some of the questions leading, with answers that did not allow a full range of responses. In some cases a simple yes or no rather than "yes, because ..." would be better. But it is what it is and should generate some interesting feedback.
"Some of the questions leading...." Yep. Hello, Response Bias! I did the best I could with this, but I can understand V-C Walker's feelings on this issue. While I hope to do better next time and don't think it's huge factor in this survey, I encourage you to ignore/omit any questions that you feel are heavily influenced.
I hope that when people post this to their local messageboards it's made clear that this is an unofficial, fan-generated poll that does not necessarily reflect the specific metrics we are currently measuring in campaign HQ. If, however, the poll seems to generate a lot of interest, we may consider having a professionally designed poll with a bit more of a scientific sampling and question phrasing based on the type of things we're interested in hearing from members of the community. I would prefer that the existing poll not be purported as coming from Paizo, though, so please keep that in mind when spreading it around. Thanks!
If and when Paizo does conduct such a survey (sounds great!), I hope it will be done in the same spirit as game design—open and transparent to all of the Pathfinder Society community. Fan input has been a great source of strength for Paizo, and will continue to be so. I choose to think that their results will mirror much of what we will see with these results.
And next...
Oh yeah...a *facepalmkeyboard* moment. Remember this question?
The most important part of Pathfinder Society play to me is: (rank in order of priority, 1 is of highest priority, 5 is lowest):
It lead to the following...
Priority range 1-5 with 1 high: Eek, I am so used to surveys that have 5 high that I nearly answered those Qs the wrong way round.Of course, now I'm nervous that I actually *did* do it the wrong way around...
I agree that it has the potential to do so. And, Pain, you know I love ya, man, but those questions probably ought to be thrown out due to bad execution.
So...one of the questions may be screwed up. How screwed up? I dunno yet. I used the standard 1 to 5 scale that we used at Wikipedia...a scale that is designed for a Global audience. "Many/some/???" people read 1 as least important and 5 as most important. (However, I'm not so naive as to think that people wouldn't have made the same mistake with the order the other way. Stuff happens.) I believe a majority did it with the right scale, however some did not.
This is understandable...and, in retrospect, I should have done the question using a different format (or done the questions individually...or with a more clear header). This might have increased the overall length (which, as stated above, can be an issue).
However, I'm not throwing this question out. Heck, it's not my place to throw it out. I believe that some of you will be able to comb through the answers for this question and find the ones that are clearly wrong and re-statify things. Some of you will want to ignore the question altogether. Please do what you wish with it and post your results on the upcoming "Survey Results" thread.
I look forward to seeing what, if anything, the Community will learn from the results or this survey. It might be a troll-augury, a Commune spell, or something more.
Since this is a meta-discussion of the survey, I wanted to highlight a couple things that worked well.
1) The survey software easily handled all the responses and can generate some basic statistical analysis. I was worried the server might pop...but maybe I'm just a worrier.
2) The survey software is actually fairly robust and allowed a lot of complexity in generating questions and trees.
3) There seems to be very few obviously redundant submissions (at least via the IP rejection filter). I will review those again to be sure that multi-submission households don't get screwed, but I'm pleased with the security against spam.
4) I'm pleased with the Community support on this. I know a lot of people pushed it along. People were happy to give feedback and share. It was nice to see.
5) I think I learned a lot during the construction and handling. I will do future surveys better.
So...constructive and thoughtful feedback?
Or...questions for future surveys?
Or...related thoughts?
-Pain
Director of Brand Strategy
|
I'd very much like to see the raw data without analysis or spin from the community. As it is, a lot of elements that I think we would find most helpful from an endeavor like this weren't included or were done in a different way than we would have preferred, but all feedback is valuable, especially when we can do our own analysis of it. Do you have any records on how thorough the polling was in terms of saturation of all geographical areas and different marketing vectors? Hearing responses is great, but getting a real scientific study is really helpful, so I hope this comes close to that.
I can't wait to see how people responded, Painlord.