| 38 people marked this as a favorite. |
Hey everyone,
I just wanted to put a thread up here to say that the amount of time you guys have been willing to spend on the surveys has been amazing, and your feedback has been great, and generally to thank you for the effort you've been putting in! It takes a lot of dedication to answer long surveys like the rules survey (or the class survey if you answered for every class), and the number of you guys who have already taken the time out of your day to answer the rules survey, even after just one day, is really impressive. It's your feedback and participation on the surveys and also here on the forums that has assisted us in getting out these extensive updates on such a quick turnaround. So thanks for all the effort you guys have put into helping us improve the playtest. Without your help, we could never make as good a game as I know we can make by following your feedback!
Do you want folks to take the non-Doomsday Dawn surveys (class/ancestry/rules/bestiary) as they are posted so you can iterate more in the play-test period or would it be better to wait until we have more experience overall and can give more through feedback?
| 4 people marked this as a favorite. |
Do you want folks to take the non-Doomsday Dawn surveys (class/ancestry/rules/bestiary) as they are posted so you can iterate more in the play-test period or would it be better to wait until we have more experience overall and can give more through feedback?
I'd say it's up to you on a topic-by-topic basis: Wait until you feel you've had enough experience with a certain topic to answer that question (and you can always go back to your survey later and fill in some more or change answers as your experiences changed). We've waited long enough on the rules survey that many of you had a fair amount of time to play the game before answering, which is good as it asks some pretty specific questions about gameplay, but you can always skip any questions you like.
Do you want folks to take the non-Doomsday Dawn surveys (class/ancestry/rules/bestiary) as they are posted so you can iterate more in the play-test period or would it be better to wait until we have more experience overall and can give more through feedback?I'd say it's up to you on a topic-by-topic basis: Wait until you feel you've had enough experience with a certain topic to answer that question (and you can always go back to your survey later and fill in some more or change answers as your experiences changed). We've waited long enough on the rules survey that many of you had a fair amount of time to play the game before answering, which is good as it asks some pretty specific questions about gameplay, but you can always skip any questions you like.
Okay great! Thanks for the response. And thank you for listening to the community and stuff.
| 11 people marked this as a favorite. |
Okay great! Thanks for the response. And thank you for listening to the community and stuff.Do you want folks to take the non-Doomsday Dawn surveys (class/ancestry/rules/bestiary) as they are posted so you can iterate more in the play-test period or would it be better to wait until we have more experience overall and can give more through feedback?I'd say it's up to you on a topic-by-topic basis: Wait until you feel you've had enough experience with a certain topic to answer that question (and you can always go back to your survey later and fill in some more or change answers as your experiences changed). We've waited long enough on the rules survey that many of you had a fair amount of time to play the game before answering, which is good as it asks some pretty specific questions about gameplay, but you can always skip any questions you like.
The playtest period has certainly been stressful and challenging, sometimes in ways that I have never experienced before (as opposed to the other designers who have all weathered an edition shift as professional game industry folks at least once), but it's really worth it to gain the wealth of knowledge that you guys have given us over the course of this playtest. We've learned so many interesting things throughout the process, often about things we never would have considered on our own. The game is going to be so much stronger thanks to all of you!
| 1 person marked this as a favorite. |
Hi Mark
Glad to be of help in perfecting the system! Kudos to you and the team for putting it all out there to be dissected by our unskilled hands... hopefully all involved will survive the process :)
One issue about the questionnaires, though - for the Doomsday Dawn questionnaire, it asks about how many minutes we spent playing. Given that I'm playing by pbp there's no real way to answer it.
Is there a possible fix (e.g. "how many posts did it take you to complete") in the works?
| 4 people marked this as a favorite. |
Hi Mark
Glad to be of help in perfecting the system! Kudos to you and the team for putting it all out there to be dissected by our unskilled hands... hopefully all involved will survive the process :)
One issue about the questionnaires, though - for the Doomsday Dawn questionnaire, it asks about how many minutes we spent playing. Given that I'm playing by pbp there's no real way to answer it.
Is there a possible fix (e.g. "how many posts did it take you to complete") in the works?
I'd recommend skipping those questions for PbP.
Thanks for listening to us and being involved! I'm so excited to see where the system goes.
Thanks for listening to all our feedback and doing posts to help keep us in the loop! My group at least is very hopeful for the future of Pathfinder 2E.
Haven't filled out that many surveys yet since my group is very behind (scheduling concerns). But I have been keeping notes and making plans to do so along with nagging my group that it'd be really helpful if they could take the time.
| 1 person marked this as a favorite. |
I don't have the time to keep up with Doomsday Dawn, but I'll continue to read the errata and blogs to stay in the survey loop. I really love Pathfinder, so of course I'll do what I can to help Pathfinder 2 succeed.
I've liked most of the changes the errata have brought, so clearly something is working.
| 1 person marked this as a favorite. |
I don't have the time to keep up with Doomsday Dawn, but I'll continue to read the errata and blogs to stay in the survey loop. I really love Pathfinder, so of course I'll do what I can to help Pathfinder 2 succeed.
I've liked most of the changes the errata have brought, so clearly something is working.
Oh man I hear you there. My group will likely finish chapter 2 tonight then skip straight to chapter 4 and probably to 7 after that. It's a very aggressive playtest schedule.
Director of Game Design
|
| 15 people marked this as a favorite. |
Hey everyone,
I just wanted to put a thread up here to say that the amount of time you guys have been willing to spend on the surveys has been amazing, and your feedback has been great, and generally to thank you for the effort you've been putting in! ....
Can/will Paizo share the results of the surveys? It would help to know if my concerns are shared by others, or if I'm an anomaly with regards to preferences.
Also, what is Paizo's confidence that the survey captures an accurate representation of entire community versus those who spend times on the forums?
We will not be posting up too much in regards to specific survey feedback and such can have a very negative impact on data collection. I think we will be more open once the playtest period is over.
Our confidence is very high, with the caveat that these are self selected surveys. That comes with a certain bias, but our numbers are getting so big, in many cases, that we feel that we are probably about as close to having an accurate picture of the community as we can get.
I will say that I am continually surprised at the difference in response between the surveys, these boards, and various social media platforms. I am sure there is a research paper in there somewhere...
We will not be posting up too much in regards to specific survey feedback and such can have a very negative impact on data collection. I think we will be more open once the playtest period is over.
Our confidence is very high, with the caveat that these are self selected surveys. That comes with a certain bias, but our numbers are getting so big, in many cases, that we feel that we are probably about as close to having an accurate picture of the community as we can get.
I will say that I am continually surprised at the difference in response between the surveys, these boards, and various social media platforms. I am sure there is a research paper in there somewhere...
While it would be fascinating to see the data, I can understand not wanting to skew the results. I hope we can get some neat graphs or something once the playtest is over, though!
I, and my group, have been happy to help refine the game. While the Playtest experience can be stressful, and I will admit that at times it can be discouraging, we are delighted to contribute to the final product.
| 7 people marked this as a favorite. |
I just wanted to chime in and say: the latest Rules Survey finally gives me the opportunity to pinpoint exactly why I like or dislike a certain feature.
I was really disappointed with the Classes Survey because it did not include any specific questions beyond the point of assessing level of satisfaction.
I don't even know how you can be drawing conclusions relative to this survey because there was very little room to explain why I rated a specific class a 1/5 and another a 5/5.
This new Rules Survey finally asks the right questions and actually investigates why a player appreciate parts of the rules and not others.
Please, keep doing the same with any future survey, I cherish the ability to give actual feedback and I don't mind sitting for 2 hours to answer them!
BTW, are you planning to have surveys that cover spells and the use of magic at some point?
I have both positive and negative feedback regarding those and I feel I have never been able to comment on these specific aspects of the rules.
They have sometimes been touched indirectly but the idea of adressing the whole theme directly feels taboo, for some reason?
| 1 person marked this as a favorite. |
I actually think the distinction of "is this class/etc powerful" and "is it interesting" is revealing.
if they are high on both, great. if high on first, but low on latter, branching out rather than doing more of what it does now is probably advisable. if low on first, but high on latter, probably it just needs power/mechanics tweaks within existing memes. if low on both, existing mechanics & memes probably are not salvageable basis and re-work of concept & mechanics is due. Obviously that's alot to infer from two minimalistic ratings, but it's an information point to consider.
Obviously any structured rating survey will have limitations, including assumptions made by survey-ee not prompted by survey itself. Open reponse as well as forum posts & other blogs & social media & other interactions do exist to accomodate ideas & communication that don't fit in structured survey.
| 6 people marked this as a favorite. |
All you guys on the playtest team have all kinds of respect from me. You're all doing such a good job remaining positive and flexible in the face of what I can only imagine is quite a lot of criticism, no small portion of which came from me.
You're doing great and I think a final product is shaping up. Hopefully we can buff out the last few sticking points for me, then I'm going to be all aboard the PF2 train.
| 4 people marked this as a favorite. |
Actually, I wanted to point out an issue with an aspect of some of the questions. For example, do I think the changes to the 1.3 Ranger are an improvement? Locally, yes, but holistically no. Yes, the changes make the Ranger better than what it was, but they don't fix the underlying problems with the class. There isn't an answer that addresses this.
There are other questions like this. Essentially I feel like I'm trapped into giving an answer that does not accurately express my opinion, but whose answer is important to what I want to convey, so I can't leave it blank.
I had the same feeling on a lot of things. Maybe not precisely about the ranger, but in a few cases I'd like an option like "I like the change but don't think it's in a good place yet", or "good direction, let's keep going this way", while the survey feels more like "yep I am happy with this", "eeeh I'm iffy about this" or "No I'm not happy with this". I often wanted to answer all three at once.
I am fairly sure from the direction of the updates that this is just a first few steps, but I can't help feeling like I'm kinda giving my blessing to something I don't fully want to settle with yet.
That said, hey, anytime you guys have a new survey, I'm ready to tick boxes! ...and write essays a few days later.
| 11 people marked this as a favorite. |
These general surveys really should be on the main playtest page. Not everyone follows blogs. Put them on the main page with the other surveys so that more people can see them.
| 1 person marked this as a favorite. |
Mass props for the surveys. Although I would hazard a guess that everyone who throws up the massive wall of negativity on the boards aren't willing to sit through the survey. Only the people who WANT this to really succeed would invest the time. As an aside... Make Dragons Great Again*!
*(By making lesser dragonkin such as wyverns or drakes for them to use as mooks, thereby allowing a dragon encounter have numbers to help combat the action economy advantage players historically have over big solo dragons).
| 6 people marked this as a favorite. |
Mass props for the surveys. Although I would hazard a guess that everyone who throws up the massive wall of negativity on the boards aren't willing to sit through the survey. Only the people who WANT this to really succeed would invest the time. As an aside... Make Dragons Great Again*!
*(By making lesser dragonkin such as wyverns or drakes for them to use as mooks, thereby allowing a dragon encounter have numbers to help combat the action economy advantage players historically have over big solo dragons).
In all fairness I’ve been fairly negative towards the playtest I’ve done every survey that has been put out, I don’t doubt there are some who didn’t do them but there are plenty of people who haven’t enjoyed the playtest who simply want to share their experiences and concern, it’s a playtest people aren’t obligated to like it, in fact if people dislike it they should make themselves heard so Paizo knows why they’re unhappy, I could be wrong but your post feels like you’re saying enjoy what’s in front of you or don’t speak.
| 2 people marked this as a favorite. |
In all fairness I’ve been fairly negative towards the playtest I’ve done every survey that has been put out, I don’t doubt there are some who didn’t do them but there are plenty of people who haven’t enjoyed the playtest who simply want to share their experiences and concern, it’s a playtest people aren’t obligated to like it, in fact if people dislike it they should make themselves heard so Paizo knows why they’re unhappy, I could be wrong but your post feels like you’re saying enjoy what’s in front of you or don’t speak.Mass props for the surveys. Although I would hazard a guess that everyone who throws up the massive wall of negativity on the boards aren't willing to sit through the survey. Only the people who WANT this to really succeed would invest the time. As an aside... Make Dragons Great Again*!
*(By making lesser dragonkin such as wyverns or drakes for them to use as mooks, thereby allowing a dragon encounter have numbers to help combat the action economy advantage players historically have over big solo dragons).
You are absolutely right. However, there are those that are openly hostile to the idea of a new edition and actively sabotage enjoyment through action or attitude. Their feedback seems less useful overall.
| 2 people marked this as a favorite. |
We will not be posting up too much in regards to specific survey feedback and such can have a very negative impact on data collection. I think we will be more open once the playtest period is over.
Our confidence is very high, with the caveat that these are self selected surveys. That comes with a certain bias, but our numbers are getting so big, in many cases, that we feel that we are probably about as close to having an accurate picture of the community as we can get.
I will say that I am continually surprised at the difference in response between the surveys, these boards, and various social media platforms. I am sure there is a research paper in there somewhere...
I have no expertise in the matter and haven't read any relevant research papers, but I will venture a hypothesis: That the population of active board posters is even more self-selected than the population of survey responders.
Thanks to you as well, both for making us a new fun game and for listening and interacting with our feedback (even though a lot of it is toxic).
Our confidence is very high, with the caveat that these are self selected surveys. That comes with a certain bias, but our numbers are getting so big, in many cases, that we feel that we are probably about as close to having an accurate picture of the community as we can get.
I get why you won’t share survey results, but could you share your analysis of this? I can think of a great many issues (and a few solutions) to this problem, and as a statistics enthusiast I’d like to hear your approach to it (not to mention seeing a high level thorough analysis will raise my already high confidence in the team)
|
Director of Game Design
|
| 7 people marked this as a favorite. |
Thanks to you as well, both for making us a new fun game and for listening and interacting with our feedback (even though a lot of it is toxic).
I get why you won’t share survey results, but could you share your analysis of this? I can think of a great many issues (and a few solutions) to this problem, and as a statistics enthusiast I’d like to hear your approach to it (not to mention seeing a high level thorough analysis will raise my already high confidence in the team)
Our confidence is very high, with the caveat that these are self selected surveys. That comes with a certain bias, but our numbers are getting so big, in many cases, that we feel that we are probably about as close to having an accurate picture of the community as we can get.
Well, there is no way that we can avoid the problems of self selecting bias in a playtest like this, but we do have a very good idea of the rough pool size for the playtest as a whole. In some of the surveys, we have reached a very high percentage of that whole (compared to what you would need for a normal random sampling to hit a 5% MOE).
Of course, we cannot be 100% certain, but some of the numbers are large enough that the remainder of the audience would have to be almost conspiratorial in group opinions to move our numbers by too large a degree.
Another example of the awesome responsiveness of the team, and one of the many reasons I love PF
So you calculate the percentage only in relation to playtest players? Or do you also try to get a picture of which parts of the general PF community participate? You obviously can’t measure the response of people who haven’t read the playtest to it, but it’s likely that those who have are more system savvy than the general public (as they are willing to learn a new, transitory, changing system).
In other words, are you also addressing the selection bias of the pool from which you are gathering, with selection bias, from?
|
Director of Game Design
|
| 4 people marked this as a favorite. |
Another example of the awesome responsiveness of the team, and one of the many reasons I love PFSo you calculate the percentage only in relation to playtest players? Or do you also try to get a picture of which parts of the general PF community participate? You obviously can’t measure the response of people who haven’t read the playtest to it, but it’s likely that those who have are more system savvy than the general public (as they are willing to learn a new, transitory, changing system).
In other words, are you also addressing the selection bias of the pool from which you are gathering, with selection bias, from?
Thats a little deep down the rabbit hole for us I think. We have some data about overall depth of participation when we ask questions about a player or GMs overall experience with Pathfinder, which lets us do some extrapolation into a wider pool, but I get a lot less certain about those numbers.
We work with the data we have, which is a far cry better than just taking guesses in our offices and hoping everybody likes it. :)
That sounds like a healthy amount of caution.
Thanks again for the responses :)
In all fairness I’ve been fairly negative towards the playtest I’ve done every survey that has been put out, I don’t doubt there are some who didn’t do them but there are plenty of people who haven’t enjoyed the playtest who simply want to share their experiences and concern, it’s a playtest people aren’t obligated to like it, in fact if people dislike it they should make themselves heard so Paizo knows why they’re unhappy, I could be wrong but your post feels like you’re saying enjoy what’s in front of you or don’t speak.Mass props for the surveys. Although I would hazard a guess that everyone who throws up the massive wall of negativity on the boards aren't willing to sit through the survey. Only the people who WANT this to really succeed would invest the time. As an aside... Make Dragons Great Again*!
*(By making lesser dragonkin such as wyverns or drakes for them to use as mooks, thereby allowing a dragon encounter have numbers to help combat the action economy advantage players historically have over big solo dragons).
Not saying that at all :) Expressing discontent (In a constructive manner) is how we all get new content. I just feel that there's enough non-constructive criticism and a few of us need to just give a few atta' boys every now and then. Deadmanwalking's threads are some of my favorite, and he(?) pokes plenty of holes in the math of the system. Which just makes the game better.
What I was trying to say was that I feel that the people who would have responded the most negatively to the survey have already dipped out of the playtest, which in my head would naturally skew surveys to being relatively positive.
I'm rather curious as to just how many surveys are being completed. At least an order of magnitude. Is it hundreds? Thousands? Tens of thousands? Hundreds of thousands? I doubt it'd be millions, but I just really don't know the size of the player base at all, let alone those participating in the playtest. Would it be possible to share that info? I can understand if it's not something you want to share though.
|
| 6 people marked this as a favorite. |
In general, I'm pleased good data is being provided and happy to help.
But I must admit I'm inclined to worry at least a bit that many of my own primary complaints aren't really covered by the surveys at all, being focused on math issues more than conceptual ones. For example, the Bestiary survey didn't address monster power levels/math much at all, which is actually the primary complaint I've seen in terms of monsters (in addition to being my own issue).
Now, maybe that's just because the folks at Paizo have already diagnosed those issues thoroughly and come up with solutions (indeed, there's pretty good odds of that), but it's still vaguely worrying in my more paranoid moments.
Deadmanwalking's threads are some of my favorite, and he(?) pokes plenty of holes in the math of the system. Which just makes the game better.
'He' is the correct pronoun, yes. And thanks, I try to be constructive in my criticisms.
What I was trying to say was that I feel that the people who would have responded the most negatively to the survey have already dipped out of the playtest, which in my head would naturally skew surveys to being relatively positive.
This does seem to me to be a potential issue, but I'm not sure there's a good way around it, unfortunately.
I'm curious if any of the results have surprised you?
| 1 person marked this as a favorite. |
No problem. I just feel bad for anyone who got stuck reading my essay length responses to some of the open survey questions.
Hah. Same here. They said be concise, but some of these questions are just so expansive that they required walls of text to answer. Like the one about what items we have problems with. That's a long list. I had over a dozen items, and all had at least a sentence or two of describing the issue. And that was with reducing many of them to categories instead of talking about every member of that category separately. I do worry about that my long answers will lead to reduced impact as it's just skimmed over to get through more responses.
| 2 people marked this as a favorite. |
BTW, are you planning to have surveys that cover spells and the use of magic at some point?
I have both positive and negative feedback regarding those and I feel I have never been able to comment on these specific aspects of the rules.They have sometimes been touched indirectly but the idea of adressing the whole theme directly feels taboo, for some reason?
This is exactly what I was thinking while filling out the rules survey. "Great, now I can finally talk about magic!"
... except no I can't. Pretty much everything else was asked, but there has been no real room in this to talk about satisfaction with the magic system now.
It really hit me a couple days ago as I was making a character for dd3, which we're running soon. Picking out spells quickly turned into an exercise in "wait, that's all this does now?" Over, and over, and over. It feels pretty clearly that they don't want you to play a buff focused caster anymore.
I'm hoping they do a survey on this at some point, because I've seen a lot of that reaction all over the place.
BTW, are you planning to have surveys that cover spells and the use of magic at some point?
I have both positive and negative feedback regarding those and I feel I have never been able to comment on these specific aspects of the rules.They have sometimes been touched indirectly but the idea of adressing the whole theme directly feels taboo, for some reason?
This is exactly what I was thinking while filling out the rules survey. "Great, now I can finally talk about magic!"
... except no I can't. Pretty much everything else was asked, but there has been no real room in this to talk about satisfaction with the magic system now.
It really hit me a couple days ago as I was making a character for dd3, which we're running soon. Picking out spells quickly turned into an exercise in "wait, that's all this does now?" Over, and over, and over. It feels pretty clearly that they don't want you to play a buff focused caster anymore.
I'm hoping they do a survey on this at some point, because I've seen a lot of that reaction all over the place.
Unfortunately, you will notice that my question has gone unanswered.
We know for that a fact that some of the devs have read this thread because they posted here.
The fact that they did not answer my question about surveys relative to magic, which I have asked for more than once, is telling.
I don't think it's coming and it might be the reason why I don't play 2nd edition when it releases.
Some of my surveys' answers have been quite positive because there is more than one thing I like in this new version of Pathfinder and some others I feel can easily be fixed in due time.
That's not the case with magic, however, and I have yet to see anything in the errata or on these boards that tells me that they intend to even ask us how we feel about it.
Ultimately, it's their game.
If they decided that they're happy with their current version of spells and magic and they don't want to know if the playerbase is happy about it too, I can't do much to change their mind.
It's such a hotly debated thing on these forums that I don't honestly see how they could have missed the fact that people are dying to share their views about it and feel that their feedback is taken into account.
In the end, it just makes me sad that I will eventually have to move on to another game when the playtest finishes.
It's even worse knowing I might not be the only one doing so because we never had a chance to officially make our voices heard.
It feels like they're dismissing any kind of negative feedback regarding spells and magic and it's kinda hard to not feel let down about it.
If I don't see it, if I don't acknowledge it, it doesn't exist.
If you have surveys, it becomes harder to pretend the issue might not be real.
I mean, the whole idea of having only one proficiency scale for all game systems is up for review but magic and spells are not ??!!
| 5 people marked this as a favorite. |
Its funny how folks get so riled up over this stuff. I am surprised the devs are so calm by comparison. This thing is their baby and they are taking everything in stride.
Good job, Paizo!
| 1 person marked this as a favorite. |
Folks...
There will be a survey targeting spells and magic specifically.
We are trying not to release too many all at once because we get a lot less responses when we do.
It's very good to finally know for sure that this will happen, thank you.
Its funny how folks get so riled up over this stuff.
Waiting 2 months for an answer will do that to some people and I am totally willing to acknowledge that I am not a very patient individual. :)
Folks...
There will be a survey targeting spells and magic specifically.
We are trying not to release too many all at once because we get a lot less responses when we do.
Great, thank you. :)
One upside of waiting a bit on that is that we can get to the higher level spells in the playtest scenarios. People going through those more or less on schedule haven't had a chance to get to high level magic yet.
Folks...
There will be a survey targeting spells and magic specifically.
We are trying not to release too many all at once because we get a lot less responses when we do.
Great, thank you. :)
One upside of waiting a bit on that is that we can get to the higher level spells in the playtest scenarios. People going through those more or less on schedule haven't had a chance to get to high level magic yet.
I'm willing to bet thats probably part of the rationale for placing the spells survey later.
| 2 people marked this as a favorite. |
I don't blame dsoinette for raging up until now. The devs has many opportunities to say there would be a Magic Survey, even going months back.
Waiting until the situation boils before saying something helpful that was already decided months ago is gonna result, well, in boiling.
Excited for that Survey though!
| 2 people marked this as a favorite. |
I will say, though, from someone with a background in how to write (and analyze) survey questions, the devs need to seriously work on writing their survey questions. Without imputing nefarious intent, it is very clear (to me, at least) the expected response to most of these questions, and I would guess that the survey results are biased because of this.
That was extremely obvious in the earliest surveys, especially the one about classes. It still shows in these most recent surveys and it's quite clear what you're expected to answer relative to each subject.
Each question feels more like wanting validation than objective feedback.
For example, you can answer that you would prefer not to have a single proficiency system for all different game elements.
It's great that this is an option.
However, the rest of this section in the survey will only ask questions that are relevant if and only if you agreed that you want to have a single proficiency system for all game elements in the first place.
If you don't want that, none of the other questions will let the devs know how you would like something else implemented.
These last two surveys are better honestly.
Yes, the bias still shows and it's often not possible to clearly choose an answer that truly represents your opinion but I can see that a lot of efforts went into including a wider ranger of options and possible feedback, which I am truly happy about.
Not everyone has a background in statistics and I can understand it being a difficult job to produce a solid and unbiased survey.
I had so much trouble with that by the end of my graduate studies that I deliberately chose a subject that would allow me to go with qualitative surveys, which are much more opened and investigative! (something totally impossible to do when you have more than 20 respondents per interviewer BTW)