Reasons why I no longer trust Paizo to produce 2E

[tivadar27]

EDIT: As someone pointed out, this is more a discussion of trust rather than good-faith (outside of point #3 below). I apologize for using the wrong term here, and have changed the title.

This came up in the forums, and I think it's only fair to state, from my perspective, why Paizo no longer has trust of their product from me. For me, that means that about 6 months ago, if they had released 2e directly, I probably would have bought it sight-unseen, and trusted that Paizo made the right decisions based on good feedback. I will not be buying 2e sight unseen after seeing the Playtest and also seeing Paizo's general response/handling of it. And, as a precursor, yes, I know I'm only one person, and this is only my feedback. I mean, what else could it be? On to the list:

1. Layout/Readability of the Playtest book: In my opinion, this is a mistake that should have been easy to detect prior to publishing. The Playtest is a Playtest, yes, but it's also Paizo's opportunity to make a first impression with their new system. Otherwise, there wouldn't have been all the hype and spoilers leading up to it. They are *marketing* this, make no mistake. Any quick scan by an outsider should have told them that the Playtest book was extremely unwelcoming, particularly to newbies. It should not have reached print in that form.

2. Quality of the surveys/conclusions drawn from them: The surveys, at least up to part 4 of the playtest, were extremely incomplete in the questions they asked to gather structured feedback. In my opinion, this lead them to infer things from their data that, bluntly put, were false. For two quick examples, Jason Buhlman commented that they were looking at remaining silver after character creation in Part 1 to infer if you had enough starting money... They didn't think (or think to ask) that remaining silver might have indicated that buying basic items was too tedious and that was the underlying cause of people having extra wealth. Second, Jason Buhlman, in that same interview, indicated they were going to look at character creation time over the course of the adventures to determine the learning curve. There's both no questions ensuring that someone who played part 4 actually played part 1, and no questions asking how much time it took to create the base character vs. leveling up. These are just two examples, but I'm sure there's more, and the surveys, in a lot of cases, didn't ask for opinions on the various core systems of the game. It showed a huge gulf, in my opinion, for Paizo to be able to infer reasonable information from their playtest data.

3. "Confirmation bias" and inauthentic responses on the forums: I'm not naming names here, but I've seen Developers handwave valid concerns regarding the game on these forums, indicating that they have playtest data. I've also seen them give responses like "well, you just don't understand/you just need to look at it a different way". Given #2 above, this is deeply concerning to me, and not just because of the tone of these posts. They seem to be taking the surveys as their One True Data, and ignoring/not giving weight to long-time customers/general consumer sentiment from people invested in their game. There are many concerns that have been raised repeatedly on the forums that haven't even warranted a comment.

4. Declaration of, and no indication of considering changing, their "1 year" playtest: It's actually 9 months, realistically, before they need to ship to the publisher. And that's not enough time. I said this upfront prior to the playtest being released, and it's even more evident given the number of holes in the playtest itself. They've essentially pinned themselves into a corner and at least on the surface, seem unwilling to bend from that. Perhaps there are marketing forces, and perhaps they *can* make multiple changes (and playtest *those* changes) to the system to address a lot of this, but printing before ready and having to errata their CRB for 2e significantly seems like a huge mistake.

These are my 4 primary reasons why I've lost faith in Paizo to produce a good 2nd edition. Note that I said *nothing* about the underlying 2e system. Yes, I think there are problems there, though I do like the core system, but if Paizo had handled the above correctly, I'd be confident in their ability to address these prior to release. To people suggesting that I should just trust Paizo, well, essentially the above is why I no longer do.

[Erpa]

You have come to the public quorum, ascended the dias and spoke your piece. To what end?

Do you wish to invite conversation?
Do you wish to bring people to your POV?
Do you wish to sow discord and have others rise up with you?
Do you wish to be convinced otherwise?

If you have come to shout to the wind your displeasure, then kudos. There is no conversation to be had.

[PossibleCabbage]

Why is "faith" important about this. If the game is fun, play/buy it. If it's not, do not. If you feel like "trying to make the game better" is pointless, stop. If you feel like "trying to make the game better" is worthwhile, do so.

[tivadar27]

Erpa wrote:

You have come to the public quorum, ascended the dias and spoke your piece. To what end?

Do you wish to invite conversation?
Do you wish to bring people to your POV?
Do you wish to sow discord and have others rise up with you?
Do you wish to be convinced otherwise?

If you have come to shout to the wind your displeasure, then kudos. There is no conversation to be had.

Honestly, I wish Paizo to address/speak to some of the 4 points I've made above. Most/all of them have been mentioned elsewhere, and to my knowledge, #1 is the only one that's been spoken about at all.

And yes, I wish to invite conversation, and see who else might have similar concerns. But your tone suggests that all you want is to belittle my post. That's your prerogative, but no, I've at no point tried to suggest I'm trying to "sow discord". To some extent, that discord already exists, from what I've seen, and I'm trying to provide *my* reasons for why it might.

[tivadar27]

PossibleCabbage wrote:

Why is "faith" important about this. If the game is fun, play/buy it. If it's not, do not. If you feel like "trying to make the game better" is pointless, stop. If you feel like "trying to make the game better" is worthwhile, do so.

Trust of a consumer base of a company seems important to me... brand name and consumer loyalty are definitely a thing. Perhaps "faith" isn't the perfect fit of a word here, granted. The playtest is supposed to allow for feedback, and honestly, there's no other reasonable place to put feedback such as this, which is much more a meta-level concern. So yes, I am trying to make the game better...

EDIT: Note, if you think I enjoyed posting this, you're dead wrong. I've been a loyal customer of Paizo for 6 or so years now. I've been sitting on this for a week or two now, and finally decided it was worth saying if I tried my best to be objective with it. I did, perhaps it wasn't perfect, and if so, that's on me, but I do feel the issues I have are relevant feedback.

[Ikos]

I think you're confusing lack of trust with lack of good faith. The latter infers dishonesty while the former means you have reservations regarding the designer’s choices. If the playtest is causing this level of personal angst and you truly believe the designers are actively out to defraud their customers, it actually might be healthiest to step away.

[tivadar27]

Ikos wrote:

I think you're confusing lack of trust with lack of good faith. The latter infers dishonesty while the former means you have reservations regarding the designer’s choices. If the playtest is causing this level of personal angst and you truly believe the designers are actively out to defraud their customers, it actually might be healthiest to step away.

You know what, you're entirely correct. I apologize, and I've edited the start of my post to say as much.

[PossibleCabbage]

If there is one thing I know to be true, it's that on the internet "individual consumers" think they are far more important than they actually are. If you're not having fun in the playtest (and a playtest is not supposed to be fun, it's supposed to be useful) then step back and check out the game when it's done.

If there are a few things I am confident in it's that the designers are listening even if they aren't engaging in the back and forth and that changes you are likely to see *during* the playtest are likely to be incremental whereas the significant changes are most likely to occur between the playtest ending and the CRB being finalized.

[Jason Bulmahn]

Folks, I have a stream here in less than an hour, and I want to make something clear.

Concerns like this are important to us, but accusing us of acting in bad faith and then demanding answers leaves us in an untenable position. Either we address you directly and then have to deal with the next customer who uses this tactic to get answers again and again, or we ignore you and play right into your theory.

So...

I am not sure how to resolve this right now and will revisit this later. Until then... I am pointing this out to moderation to handle.

[AndIMustMask]

it's kind of funny that we get another thread of someone voicing their complaints on something, and the first reply is "well why are you even here then?"

[Excaliburproxy]

Erpa wrote:

You have come to the public quorum, ascended the dias and spoke your piece. To what end?

Do you wish to invite conversation?
Do you wish to bring people to your POV?
Do you wish to sow discord and have others rise up with you?
Do you wish to be convinced otherwise?

If you have come to shout to the wind your displeasure, then kudos. There is no conversation to be had.

I am guessing he is venting and wants to be heard. *shrugs*

I dunno.

Towards OP's points:

1. I also don't like the layout of the book and I feel like judicious reprinting of rules in multiple places and listed page numbers for listed rules will go a long way in fixing my issues with the rulebook. That said, that may not be the absolute end-all-be-all to format design.

2. I also felt like my actual concerns were being addressed when I was answer questions like "how many resonance points did you have left" and the like. Like: of course I had spell and resonance resources left at the end of the day. That does not mean we did not have to end the day early when we were getting low. That does not mean that managing that resource was fun or unstressful. This is coming from a guy who LIKES resonance! That said, maybe there are appropriate ways to use that kind of data but I can see a lot of people wanting to tell Paizo how they feel about a mechanic and then only being able to answer a potentially meaningless number.

I--for one--wish that Paizo would ask if you felt like you had enough feats, for instance. It's not that I don't have enough option, it's that I don't have enough slots to make the builds that seem fun to me.

3. Eh. This is hard but I think you are being to harsh on the designers here. I think the designers have been thinking very hard about this game for a long time and having these same arguments among themselves and in the Paizo office more generally. As such, they already feel like they have resolved some of these issues and/or feel like people in general will just get to the same point they have if they talk through the issues more among themselves.

4. This game needs to be done some time and no game will ever be perfect. I support Paizo's decision to pick a date and stick to it rather than just take in feedback forever until fans stop being mad. That is how we got the grey goo that is 5e DnD.
-w-

[tivadar27]

Jason Bulmahn wrote:

Folks, I have a stream here in less than an hour, and I want to make something clear.

Concerns like this are important to us, but accusing us of acting in bad faith and then demanding answers leaves us in an untenable position. Either we address you directly and then have to deal with the next customer who uses this tactic to get answers again and again, or we ignore you and play right into your theory.

So...

I am not sure how to resolve this right now and will revisit this later. Until then... I am pointing this out to moderation to handle.

I apologize to you regarding this. As I've stated, I used the wrong term initially and have adjusted my post accordingly, it's a matter of trust, and not good-faith. If you determine this to be inappropriate, that's you're call. I'll respect your decision.

[Sara Marie]

Folks, I'm going to address not only this thread, but some of what happened in other similar threads since this one is just starting.
It would be great if we could assume people within the community on paizo.com are here in good faith. That does not mean everyone has to agree or like everything, but that the assumption should be that people are being genuine when they speak about their feelings or opinions. That includes assuming people are asking follow up questions out of curiosity and interest in why another person feels a particular way. If you want to respond to something, and you speak in a way that assumes they are posting in good faith, you can help keep the thread on track and reasonable. Trying to rephrase what other people are saying in an inflammatory manner is not helpful, it just escalates things into a giant derail of arguments. If you feel someone is posting in bad faith, it is not appropriate to call them out on our forums. Please send relevant information to community at paizo.com.

[Sara Marie]

I've also observed that with subjects where multiple threads have been started, often the same voices show up, posting in quick succession. If you find yourself repeating your arguments over and over and getting into the same intense debate on the same topic, consider taking a break and letting other community members who may not be online with the same frequency have a chance to read and comment before a thread goes off into the weeds.

[D@rK-SePHiRoTH-]

Jason Bulmahn wrote:

or we ignore you and play right into your theory

You shouldn't be worrying about this. You're in control. Let people think what they want. Just make sure your communication is clear when you want to send a message. That is how politicians handle controversies and they are, arguably, the most succesful category of people in the world.

[Mark Carlson 255]

I think you have valid points especially 1 and 2.
After reading the survey Q's I was very surprised to not see a simple question, "Did you have fun in this section?"

MDC

[Scythia]

As much as I don't like the direction of PF2, I don't think I'd go this far. It's not a matter of trust, just realising that they are going a different way than I would like. That's how it works sometimes in a world with different people that have different wants and ideas.

[HidaOWin]

Mark Carlson 255 wrote:

I think you have valid points especially 1 and 2.

After reading the survey Q's I was very surprised to not see a simple question, "Did you have fun in this section?"

MDC

"Fun" is not a terribly useful metric, one player might only find combats which are never challenging fun, while another player might find ones where the party TPKs fun. Fun is so variable that its almost impossible to have something that is found universally "fun". You're better off striving for a particular approach for your game and letting the audience for that game find it.

[PossibleCabbage]

One thing I think is worth considering is that while Designers might not read a thread as it happens so they can participate, it's extremely likely they will get around to reading it eventually. So if you make good points they'll be read and considered, but it's not like they need to post in a thread to have taken something from it.

[The Once and Future Kai]

tivadar27 wrote:

As someone pointed out, this is more a discussion of trust rather than good-faith (outside of point #3 below). I apologize for using the wrong term here, and have changed the title.

Thanks for editing your post. I was feeling reactive to the use of "good faith" so the edit was helpful. I also appreciate you being, generally, chill about it.

[Anguish]

HidaOWin wrote:

"Fun" is not a terribly useful metric

Strongly disagree.

In my humble opinion, "fun" is the best metric, and it should be the first, foremost question asked. Most other questions should pivot around it.

"Did you find playing your character fun?"
"Did you find creating your character fun?"
"Did you find picking your ancestral feats fun?"
"Did you find generating your scores fun?"
...
"Did you find your character's combat abilities fun?"
"Did you find your character's out-of-combat abilities fun?"
...
"On a scale of 1 to 10, how fun do you find the following new rules...?"

This is a game. It's all about having fun, and frankly how subjective "fun" is comes down to irrelevance. See, if 97% of respondents indicate that unarmed gnome barbarians wielding improvised weapons and specializing in occult crafting is "fun" while 3% of respondents indicate that human fighters with greatswords are "fun", that strongly informs the designers that it's time to work on making human fighters with greatswords "better". Sub-questions are how they could determine what about human fighters with greatswords are un-fun.

Hmmm. Turns out my opinion is perhaps less humble than I'd originally thought.

[Requielle]

I trust that Paizo is making their best effort to put out a solid game - and that they have a pretty clear vision of what they are trying to achieve. That doesn't mean I'm not worried (in a general sense) that this edition seems to be shaping up to be a fundamentally different game than PF1E was.

Some of that worry is admittedly rather selfish. I thought Paizo did a wonderful job of refining the 3.5 chassis. I remember reading through the CRB and just going "Yes! That is so much better!" over and over. I was honestly hoping for that experience again. There are certainly some ideas in the playtest that we've already stolen and houseruled into our Reign of Winter campaign - and several more that we are looking at to see if we can figure out how to make them work. But the overall system isn't fitting well with how me or my friends play - it feels restricting and controlling rather than creative.

Maybe the gaming world has moved on and I'm just TTRPGing the Get Off My Lawn AP. If the overall game design is already decided and not likely to change and that publish date is set in stone ... well, I'd probably rather be all wrong in my critiques and see Paizo succeed despite my misgivings, than to be right and have Paizo forge on ahead into a brick wall.

[TintoTenga]

Sara Marie wrote:

I've also observed that with subjects where multiple threads have been started, often the same voices show up, posting in quick succession. If you find yourself repeating your arguments over and over and getting into the same intense debate on the same topic, consider taking a break and letting other community members who may not be online with the same frequency have a chance to read and comment before a thread goes off into the weeds.

Amen.

This issue really detracts from the usability of the forum.

[plaidwandering]

I trust that the designers are putting in a lot of effort to put out what they think is a good game. I trust that they are trying very hard.

but sometimes people think their ugly baby is the cutest in the world too

If they can't see by this point that there are major flaws, then they are too close to the project to be aware of that

Right now I really fear this being a significant business failure, and minor tweaks aren't going to change that

[The Once and Future Kai]

Anguish wrote:

HidaOWin wrote:

"Fun" is not a terribly useful metric

In my humble opinion, "fun" is the best metric, and it should be the first, foremost question asked. Most other questions should pivot around it.

Well, let me quote my humble opinion.

The Once and Future Kai wrote:

I'm imagining Gary Gygax and Jeff Perren designing Chainmail...

"Let's give each unit numeric health...hit points...to show how much damage they've taken."

"I don't know Jeff. HP just isn't fun."

A rule doesn't have to be innately "fun" if it indirectly contributes to the game being interesting, challenging, etc. I think Resonance needs a significant rework but, personally, I don't care if it's fun or not as long as it makes the game better (currently Resonance does not, in my opinion).

"Fun" isn't as simple as making people feel happy. Survey results that focus solely on "fun" lack nuance and don't consider how restrictions contribute to "fun".

[Anguish]

The Once and Future Kai wrote:

Anguish wrote:

HidaOWin wrote:

"Fun" is not a terribly useful metric

In my humble opinion, "fun" is the best metric, and it should be the first, foremost question asked. Most other questions should pivot around it.

Well, let me quote my humble opinion.

The Once and Future Kai wrote:

I'm imagining Gary Gygax and Jeff Perren designing Chainmail...

"Let's give each unit numeric health...hit points...to show how much damage they've taken."

"I don't know Jeff. HP just isn't fun."

A rule doesn't have to be innately "fun" if it indirectly contributes to the game being interesting, challenging, etc. I think Resonance needs a significant rework but, personally, I don't care if it's fun or not as long as it makes the game better (currently Resonance does not, in my opinion).

"Fun" isn't as simple as making people feel happy. Survey results that focus solely on "fun" lack nuance and don't consider how restrictions contribute to "fun".

I'm not going to get into a circular whirlpool argument, but it looks like we disagree. I believe nuance comes from nuanced questions, and I believe the hypothetical conversation you posit isn't applicable as it portrays a solely internal discussion that doesn't involve the target audience. But I imagine you already know that, so as I say, we disagree, and that's okay.

[Diego Valdez]

Removed a post. Personal attacks directed at specific Paizo employees are not acceptable. Whatever your thoughts on the state of the game may be they can be expressed without resorting to personal attacks or claims of intentional malice on the part of staff members. The design team is reading and processing data from numerous sources. That a specific topic you have concerns about has not been addressed the way you want it to be doesn't mean they haven't considered it, are ignoring you, or are trying to deceive you. Suggesting such just serves to undermine the productivity of the discussion.

[tivadar27]

I'd like to reiterate this, but I am sincerely sorry, particularly to Jason, since I mentioned his name directly, for the initial title of my post. I unintentionally used words that I shouldn't have regarding the Paizo staff, as I meant something different from what I said, and people were justified to react as they did.

While I obviously have a lot invested in this game and may think that Paizo is mistepping in areas, by no means do I think that they're not trying to make a good product that their customers will enjoy, and I mistakenly said that.

[Tridus]

The Once and Future Kai wrote:

"Fun" isn't as simple as making people feel happy. Survey results that focus solely on "fun" lack nuance and don't consider how restrictions contribute to "fun".

That said, "fun" is also an essential requirement. The game must have it to be a success in the market.

The surveys ask about fun because if people en masse say the game isn't fun, that's a showstopper level problem.

[PossibleCabbage]

I think the thing is, though, that "fun" is a holistic quality not an atomic one. We want to see if the game itself is fun, or if a specific way of playing it is fun, but we can't answer "is it fun" by drilling down and looking at specific isolated mechanics, since "fun" comes from the interplay of those mechanics.

Like we can look at "I fail too much at the thing I want to be good at" as a reason something might not be fun, but that could either be from "the DCs are too high" or "the bonuses are too low" or even both.

[Doktor Weasel]

I do think these are all valid points here, but yeah, the initial wording was problematic and leads people into thinking of this as an attack, when based on your other comments, it doesn't seem like it is. It's not really an issue of 'good faith.' Clearly it's in everyone's best interest (especially Paizo's) to make the game the best possible. The big problem comes in how you determine what the 'best' is. Trust is a better term, but again might give the wrong impressions on what is meant.

Point 1: Yeah, this is a big failure, sadly. The amount of page flipping required, and unclear rules are certainly major problems that need to be cleaned up. But the playtest is really the place to do that cleaning. I'm not so sure that this should have been easy to detect beforehand. It's really hard to read through rules that you know and see how clear they are. You already know what is trying to be said, so it's hard to see the problems.

Point 2 is one I'm very concerned about myself. It's not that I think they're trying to swing things in one way or another. It's just that writing good surveys is really hard, and it's very easy to get the wrong idea from the data collected by imperfect surveys. The open response surveys do help some in this regard though. But I'm not sure how thoroughly these are being analyzed. I know my responses tended to be rather verbose, and I suspect others are too. That can be an absolute mountain of text to sort through for a small team.

Point 3 kind of ties in a bit with point 2. The "How many times did you run out of resonance?" question and it's apparent usage are a good example. The low number of PCs running out completely was pointed at a lot early on, apparently reading this as people aren't having a problem with resonance. But that's a bad metric. It doesn't take into account the disincentives resonance gave and many other things. This really wasn't helped by the fact that they responded to early criticism of resonance by just restating their reasoning behind resonance, as if we just don't understand how bad Cure Light Wounds are and how important it is to have resonance. (Although to be fair, some of this may be bias on my side, of seeing these comments as protective of resonance). But thankfully, there appears to have been a change of heart, and now they're working on dramatically paring back resonance. In a related topic, they seemed to have been big on trying to cut down between-combat healing. But eventually stopped trying to restrict it and gave Treat Wounds instead. So I think there are hopeful signs here. There is some defensiveness of their ideas, which is a pretty human reaction. But I think they're overcoming it.

Point 4 is something I've also worried about a lot myself. This really is a short test, only like 6 months before they have to dedicate everything to writing the new book to get it out for Gencon. And it seems to me that we're just getting to the point where the playtest is having major effects. But there is a lot left to look at. And some changes are just too big do be handled with the current update documents. They're running into that with the resonance changes. And some common complaints like the restrictiveness of classes and weakness of spell-casters haven't really been touched all that much yet. And they're about on the same scale as resonance. I continue to feel a second iteration of playtest rules is needed to really get things right. But there are problems with this as well. Playtesting is already pretty grueling for both players and developers, doing even more of it can really wear people down. And the test has to end sometime. So I'm not sure what the right approach is.

[The Once and Future Kai]

PossibleCabbage wrote:

I think the thing is, though, that "fun" is a holistic quality not an atomic one. We want to see if the game itself is fun, or if a specific way of playing it is fun, but we can't answer "is it fun" by drilling down and looking at specific isolated mechanics, since "fun" comes from the interplay of those mechanics.

Exactly. For instance, character creation may not be "fun" but is that because of the books layout? Which has nothing to do with mechanics. Or is it because the specific character options are poorly designed? Or is it because the character sheet is awful? Or is it because some players like learning new systems and think that's "fun", but other people do not like to learn new systems and do not think that's "fun". Or is it based on the class or ancestry? Or the player's skill level?

Not that having "fun" isn't important but polling that doesn't provide actionable data. It's like asking, "Is something wrong with the country?" You'd probably find out that a lot of people think something is wrong with the country but that doesn't provide any actionable data on how to rectify that. More specific questions built around data points, not opinions, can tease out commonalities that the respondents were not even aware of.

[LordVanya]

I basically agree with the OP.

Personally, I think it best to give the reactions of potential customers at least the same weight as your vision if not more.
After all, if the product doesn't sell, then your vision won't amount to much.

On the subject of the amount of time given to the Playtest; I'd rather have the first print take longer to come, than get it early and have to errata it into unrecognizability.
To quote Shigeru Miyamoto (though the application here isn't as impactful):
"A delayed game will eventually be good, but a rushed game is forever bad."
Fortunately, the format of a TTRPG allows for a rushed game to eventually be fixed, but that first impression is what will win new fans and customers the most.

On the subject of fun.
Everyone on one side seems to be saying it isn't important because you can't just judge things on fun.
However, no one on the other side of the argument has suggested or even implied that the sum total of every survey should solely on fun.
It is an important question in the context of developing a game, and taken in context with other more specific questions can give developers a better and more clear understanding of their players' experiences.
No developer worth his salt (including myself in that) would ignore the fun factor.
Even a horror game has to be fun.

I hope the design team takes this thread to heart.
There is more to this whole endeavor than just designing and math and vision.

[The Once and Future Kai]

LordVanya wrote:

Everyone on one side seems to be saying it isn't important because you can't just judge things on fun.

However, no one on the other side of the argument has suggested or even implied that the sum total of every survey should solely on fun.

Everyone and no one statements are not helpful. Just from this thread I can quote exceptions to these generalizations. I started to, in fact, but that seemed like a distraction from the point.

[Rysky]

LordVanya wrote:

To quote Shigeru Miyamoto (though the application here isn't as impactful):

"A delayed game will eventually be good, but a rushed game is forever bad."

I can easily see Jason flipping a kotatsu.

[The Once and Future Kai]

tivadar27 wrote:

3. "Confirmation bias" and inauthentic responses on the forums: I'm not naming names here, but I've seen Developers handwave valid concerns regarding the game on these forums, indicating that they have playtest data. I've also seen them give responses like "well, you just don't understand/you just need to look at it a different way". Given #2 above, this is deeply concerning to me, and not just because of the tone of these posts. They seem to be taking the surveys as their One True Data, and ignoring/not giving weight to long-time customers/general consumer sentiment from people invested in their game. There are many concerns that have been raised repeatedly on the forums that haven't even warranted a comment

Confirmation bias isn't just an issue for the developers. Earlier in the playtest, there was clearly a healing problem. I proposed a One Hour Healing Ritual solution that I thought was nifty and that many people seemed to like. Instead Paizo went with Treat Wounds. My reaction was "my idea was much better than this because this doesn't resolve xyz!" In the end, Treat Wounds works well for the community (except for the scaling DC issue) and neatly resolved the problem with an existing mechanic. It's a mistake to assume that Paizo isn't working to address problems just because they're not engaging customers on an individual level. Which, frankly, would be a full time job in and of itself. It remains to be seen how this data heavy approach will work out but claiming that the developers are not working to resolve issues because they're not engaging one to one with every customer is unhelpful.

[tivadar27]

The Once and Future Kai wrote:

tivadar27 wrote:

3. "Confirmation bias" and inauthentic responses on the forums: I'm not naming names here, but I've seen Developers handwave valid concerns regarding the game on these forums, indicating that they have playtest data. I've also seen them give responses like "well, you just don't understand/you just need to look at it a different way". Given #2 above, this is deeply concerning to me, and not just because of the tone of these posts. They seem to be taking the surveys as their One True Data, and ignoring/not giving weight to long-time customers/general consumer sentiment from people invested in their game. There are many concerns that have been raised repeatedly on the forums that haven't even warranted a comment

Confirmation bias isn't just an issue for the developers. Earlier in the playtest, there was clearly a healing problem. I proposed a One Hour Healing Ritual solution that I thought was nifty and that many people seemed to like. Instead Paizo went with Treat Wounds. My reaction was "my idea was much better than this because this doesn't resolve xyz!" In the end, Treat Wounds works well for the community (except for the scaling DC issue) and neatly resolved the problem with an existing mechanic. It's a mistake to assume that Paizo isn't working to address problems just because they're not engaging customers on an individual level. Which, frankly, would be a full time job in and of itself. It remains to be seen how this data heavy approach will work out but claiming that the developers are not working to resolve issues because they're not engaging one to one with every customer is unhelpful.

I'd agree that it's not only an issue for the developers. I don't deny that me 'seeing' posts that are skewed towards what I say is more likely. That being said, I've found the tone of developers in previous threads to be rather dismissive. And I've found that there's been very little said or done about some of the key issues that get raised on here repeatedly (class feat siloing and the +/-10 critical system for example). Do I expect them to immediately make a change, no, as there *are* people on both sides of this issue, but it'd be nice to have them speak to some of these issues more, and, in addition, see them reflected in the surveys, as these are issues that have come up repeatedly on the forums.

Note that it's possible they have heard and are working on these things, but just not saying anything. In that case, a *bit* more transparency here would probably go a long way.

[The Once and Future Kai]

tivadar27 wrote:

Note that it's possible they have heard and are working on these things, but just not saying anything. In that case, a *bit* more transparency here would probably go a long way.

Personally, I don't think it's a lack of transparency as much as splintered communication. The developers make regular blog posts and occasionally post on the forum, but they're also interacting with fans on Twitch, Facebook, Twitter, etc. They don't seem to have an official mode of communication that covers all the bases. I've seen questions answered on Twitter or Twitch that went unanswered despite being asked on the forums. Alternatively, I've seen answers by developers buried in forum threads. It would be helpful if they collated their responses and shared them formally so it's not detective work.

[David knott 242]

One thing to keep in mind is that we are used to Paizo telling us stuff about their new products, with our responses mostly being that we do or don't like it.

But in this case, the communication flow is mostly reversed. The playtest began with them basically saying, "Here are some things we are thinking of trying. What do you think? How well do they work for you?" The bulk of the communication is our detailed reactions from us to them.

We should not be expecting individual responses to our particular critiques -- Paizo is more interested in the cumulative information than in what any single person says. Clearly, some of the feedback they have received has driven updates to the playtest, but some of it is still being reviewed or may have an effect that they don't need to playtest (such as them concluding from the feedback that a particular new mechanic doesn't work at all and that they should revert to a tried and true PF1 method).

[Ikos]

AndIMustMask wrote:

it's kind of funny that we get another thread of someone voicing their complaints on something, and the first reply is "well why are you even here then?"

That question is actually at the crux of the matter. If someone has already decided that the playtest is not enjoyable, that they’ve lost all confidence in the developers, and that they’ve aired their concerns multiple times, but feel absolutely ignored, it seems quite natural to wonder, if one truly believes those things, why the continued dramatics? Either the issue is as dire as described (and pointless) or one of those variables is not entirely the case.

[Mark Carlson 255]

HidaOWin wrote:

Mark Carlson 255 wrote:

I think you have valid points especially 1 and 2.

After reading the survey Q's I was very surprised to not see a simple question, "Did you have fun in this section?"

MDC

"Fun" is not a terribly useful metric, one player might only find combats which are never challenging fun, while another player might find ones where the party TPKs fun. Fun is so variable that its almost impossible to have something that is found universally "fun". You're better off striving for a particular approach for your game and letting the audience for that game find it.

I agree and disagree with this; quite simply it is how you ask and where you ask did you have fun with rule X, Y or Z.

Example's:
1) I do agree that most people do not like PC death but most players and GM think it is important to the game. I say most as I know groups that treat many PnP RPG's more like PC games with re-spawn times vs traditional resurrection methods.
2) Asking if rule X is fun or add's to the game helps decide if it needs re-tooling, modifying or removing from the game so as to keep and attract customers.
Note: I do agree that GM's should make it very clear that this is a beta test and to expect issues and try and have that not affect their over all impression of the possible finished product.

Simply asking if core rule's X, Y, Z are fun also gives very important feedback to the dev team (if the data gathered can be guaranteed accurate and not biased in any way). For example I know of dev's that love a rule or rules and it works great in their games but when it is brought out to other's the feedback they get is others do not like it in any way. Some dev's have an easier time of getting rid of the rules and or redesigning from the ground up vs others.
Note: Again accurate data and asking the right questions in an un-biased way is very important.
MDC

[Doktor Weasel]

tivadar27 wrote:

I'd agree that it's not only an issue for the developers. I don't deny that me 'seeing' posts that are skewed towards what I say is more likely. That being said, I've found the tone of developers in previous threads to be rather dismissive. And I've found that there's been very little said or done about some of the key issues that get raised on here repeatedly (class feat siloing and the +/-10 critical system for example). Do I expect them to immediately make a change, no, as there *are* people on both sides of this issue, but it'd be nice to have them speak to some of these issues more, and, in addition, see them reflected in the surveys, as these are issues that have come up repeatedly on the forums.

Note that it's possible they have heard and are working on these things, but just not saying anything. In that case, a *bit* more transparency here would probably go a long way.

I'd like more transparency too. But it is tricky. It's been said several times that they don't want to taint the responses to the playtest and they don't want to give a false impression that something is going to happen when they're simply considering it. And these are valid concerns. But that said, it would be nice to know if certain things are even being thought about, or are what they want them to be. Examples I can think of are the +/-10 crits, magic weapon damage dice, restrictive classes, the greatly reduced magic, the current state of alchemists etc. There are hot topics, but I haven't seen any mention from the devs if they're even considering changes, let alone what they might be. Can they talk more without raising false hopes or altering the playtest? I don't know. We have assurances that everything is on the table and that they follow the threads here, but we're mostly in the dark. It's no surprise that some people will interpret the silence to mean things won't change. Lack of knowledge helps bread fear.

They did give some heads-up that ancestry changes were being looked at for a bit, and now we've got at least the first round of that. Even resonance had a few hints too, but was mostly opaque before the change was announced.

[Requielle]

Ikos wrote:

AndIMustMask wrote:

it's kind of funny that we get another thread of someone voicing their complaints on something, and the first reply is "well why are you even here then?"

That question is actually at the crux of the matter. If someone has already decided that the playtest is not enjoyable, that they’ve lost all confidence in the developers, and that they’ve aired their concerns multiple times, but feel absolutely ignored, it seems quite natural to wonder, if one truly believes those things, why the continued dramatics? Either the issue is as dire as described (and pointless) or one of those variables is not entirely the case.

You've conflated a lot of things there. Someone doesn't have to sign up for the all-you-can-dislike buffet to decide that the playtest is not enjoyable.

[Lord Norin]

I also agree with the OP's sentiment... Not only does the game not feel like Pathfinder 1E (Epic Magic, the Joy of theory crafting) but it is not fun to play... My group voted to end our playtest after episode 3 - Nobody was having any fun and voted to play 5E.

Jason there are some things I do like about 2E.. the three action economy, critical on +/- 10 and the variable success states for saves vs. various effects. But these few things do not make up more the lackluster feats, skills and spells... Everything has been dialed back... That is not Pathfinder, at least not my pathfinder... I want Powerful Magic, I want fighters who are more than there weapons magical pluses, the proficiency system is klunky and insulting, I liked that in PF1 the rules were the same for PC's and NPCs it made building them more science and less art, thus giving me more time to work on story and theme....

I respect that you want to dial down the crazy numbers but by making everyones skills progress at more or less the same rate all you have done is homogenize the characters so that their choices and agency are less impactful. Which is not Pathfinder at all.

Pathfinder is all about choices, it is the legos of RPG's, when you reduce peoples choices and limit what there characters can be... especially compared to PF1's Core Rules well, I think you can see why the citizens are going for the torches....

I for one want PF2 to be awesome, as a Paizo Fan, I have purchased every pathinder and Starfinder book and most of the map packs and what not.. At this point if this was the final product I can't say I see that pattern continuing...

With the exception of today's podcast I have watched each Q and A session and in general not been impressed with the transparency with which you guys answer questions about what is working and what you guys are considering tweaking, and in no instance have I heard mention of the absolutely atrocious nerf job to spells been mentioned, yes some spells needed serious revisions.. but others like Unseen Servant and Mage Hand... these needed no changes and you nerfed than anyway...

I don't believe that the current design team has it's pulse on what the current pathfinder fans want.. Nor am I certain you have a feel for what the rest of the market wants... I can tell you this however, the game. in it's current state is not fun...

As a game designer myself, I can understand how easy it is to become blind to flaws, which is why you conduct outside playtests... which I have done, for games that have failed on Kickstarter (see Fields Tactics and TableFlip for the curious) and know that you learn more from failures than from success, so understand that when I tell you when a lot of people are telling your game is not fun... You listen! find out why they feel that way.. check that against other data and find out if it is a bias or a legitimate feeling...

Sorry for the length of this post, I really wanted to get this off my chest. I love Pathfinder, I really think the AP's are amazing, and well written and fun to play, but this is just not up to that legacy... The adventure is lackluster, it feels more like a D&D encounters scenario than a Paizo episode... And the rules just plain are not working for me and a significant amount of the current fans.

I will continue to lurk and post here and there but otherwise unless their is a significant rules revison, I can't see my group returning to the playtests or picking up PF2.

[Shisumo]

tivadar27 wrote:

2. Quality of the surveys/conclusions drawn from them: The surveys, at least up to part 4 of the playtest, were extremely incomplete in the questions they asked to gather structured feedback. In my opinion, this lead them to infer things from their data that, bluntly put, were false. For two quick examples, Jason Buhlman commented that they were looking at remaining silver after character creation in Part 1 to infer if you had enough starting money... They didn't think (or think to ask) that remaining silver might have indicated that buying basic items was too tedious and that was the underlying cause of people having extra wealth. Second, Jason Buhlman, in that same interview, indicated they were going to look at character creation time over the course of the adventures to determine the learning curve. There's both no questions ensuring that someone who played part 4 actually played part 1, and no questions asking how much time it took to create the base character vs. leveling up. These are just two examples, but I'm sure there's more, and the surveys, in a lot of cases, didn't ask for opinions on the various core systems of the game. It showed a huge gulf, in my opinion, for Paizo to be able to infer reasonable information from their playtest data.

3. "Confirmation bias" and inauthentic responses on the forums: I'm not naming names here, but I've seen Developers handwave valid concerns regarding the game on these forums, indicating that they have playtest data. I've also seen them give responses like "well, you just don't understand/you just need to look at it a different way". Given #2 above, this is deeply concerning to me, and not just because of the tone of these posts. They seem to be taking the surveys as their One True Data, and ignoring/not giving weight to long-time customers/general consumer sentiment from people invested in their game. There are many concerns that have been raised repeatedly on the forums that haven't even warranted a comment.

Mark Seifter spent some time tonight on the Twitch stream talking about how the dev team is actually using the playtest data, and how it relates to forum information.

[tivadar27]

@Shishumo: Just saw this, the one thing that stuck out is Jason's comment that they're at least looking at archetypes related to combat styles (I'm assuming something like two-weapon fighting). That addresses a bit of the siloing concern, which I appreciate.

Regarding the surveys, they've addressed some questions, but the problem largely is about questions they *didn't* ask, and that's a large part of the problem. Also, I'm totally on board with Linda explaining questions to 5 year olds!

[LordVanya]

The Once and Future Kai wrote:

LordVanya wrote:

Everyone on one side seems to be saying it isn't important because you can't just judge things on fun.

However, no one on the other side of the argument has suggested or even implied that the sum total of every survey should solely on fun.

Everyone and no one statements are not helpful. Just from this thread I can quote exceptions to these generalizations. I started to, in fact, but that seemed like a distraction from the point.

My "everyone" is a very small number of people.

I didn't mean it to be taken as everyone at large, but I thought that was a giving within the space of a single one-page thread.

[LordVanya]

The Once and Future Kai wrote:

tivadar27 wrote:

Note that it's possible they have heard and are working on these things, but just not saying anything. In that case, a *bit* more transparency here would probably go a long way.

Personally, I don't think it's a lack of transparency as much as splintered communication. The developers make regular blog posts and occasionally post on the forum, but they're also interacting with fans on Twitch, Facebook, Twitter, etc. They don't seem to have an official mode of communication that covers all the bases. I've seen questions answered on Twitter or Twitch that went unanswered despite being asked on the forums. Alternatively, I've seen answers by developers buried in forum threads. It would be helpful if they collated their responses and shared them formally so it's not detective work.

It might behoove them to do a q&a blog post from time to time featuring the most debated topics from all their ongoing discussions.

Though, I get the impression the staff is spread kinda thin as is.