Why is playtesting considered more important than other forms of analysis?


Advanced Class Guide Playtest General Discussion

1 to 50 of 167 << first < prev | 1 | 2 | 3 | 4 | next > last >>

20 people marked this as a favorite.

Reading through the new forum here on the ACG, I can't help but pick up on a strong undercurrent that seems to be just smiling and nodding at all the theorycrafting and other analysis and looking only at playtesting reports, or at the very least, considering playtest reports as being somehow more valuable.

I have some problems with this attitude:

1) Playtesting is always going to be a case of small sample sizes, at the very least until dozens of people play entire year long campaigns as these classes. Your experience playtesting in a short session (or even a dozen) will be tainted by luck and circumstance. If you roll really well for the whole game, you're going to have a very different opinion of the class than if you rolled poorly, or even average.

Other analysis will pick up on this. Instead of seeing, "oh, I rarely missed with my Hunter," they'll look at the actual math and point out that they are behind other 3/4 BAB characters in the accuracy/damage department because their main class buff duplicates an item the others will have (and it doesn't matter that they don't have to buy the item, because there won't be anything accuracy/damage related to spend that money on anyway).

2) Playtesting is inherently biased. Most people are only going to playtest a class they already think looks good to them. You're basically going to get a lot of noise, then, from people who love the concept of their characters and so don't really care about the mechanics behind them, which is really the whole point of a playtest like this. I saw one playtest report, for example, from a Warpriest of Gorum who was thrilled by the character and had tons of fun, even though literally everything he described doing would have been possible for a base Cleric as well. Non-playtest feedback is valuable in this regard because it is "pure." Someone who picked a Hunter because they love animals and spent the whole playtest loving their pet tiger wouldn't necessarily know or care that a Druid (or Ranger with an archetype) could have the exact same pet with other, stronger abilities. Such a player will make Hunter look perfectly fine when it is clearly not in good shape.

On a similar note, you can use volume of comments as a measure of success to some degree here, then. Just look at the Skald discussion (probably the most problematic class in the playtest)--it's tiny compared to the others because the mechanical flaws were already pointed out fairly quickly and the flavor is just not grabbing people. In fact, the main playtest complaints coming out of it are things like "Scribe Scroll doesn't feel right for a viking!" That's because the only people playtesting the Skald are going to be people playing with lots of characters that could benefit from the Rage--you'll never get a report from a Skald in a party with a Swashbuckler, Wizard, Cleric, and Inquisitor because anyone can immediately see that the Rage Song will be utterly useless for them. So, you won't see the complaints that say, "Rage song was pointless! Nobody in the party wanted it!" that just about every analyst and theorycrafter immediately picked up on.

3) PFS is the realm where the rules and mechanics matter most, but almost nobody plays that way outside of PFS. For example, if I were to run a playtest for these classes, the Hunter and Warpriest would look much better because I don't use any magic items at all, so their ability to add enhancement bonuses to stats/weapons/armor would be far more valuable than it really is in an official environment like PFS (where it basically just means "this class has slightly more money").

I prefer lower levels, too, and deliberately avoid all of the "one bad roll and you die" abilities (like instant death or petrification or whatever) because I don't allow coming back from the dead. So, anything my players had to say about, for example, a Swashbuckler, will be tainted by that preference. They'll see how fun it is at low levels (well, levels after 1, since they suck for a whole level), and never get to the point where they're failing instant death saves left and right because they're the "front line muscle" but have only a good Reflex save.

In other words, table variation and houserules are everywhere in the community. I think that's a great thing in and of itself, but it makes a public playtest very difficult at best. Most people have used their houserules so long that they don't even realize it's a houserule anymore. You're going to get very uneven reports based on that, and it's going to make the situation look better or worse than it will really be in PFS, which is the main place the rules matter and where they need to be the most "crisp."

The point, I suppose, is that non-playtesting analysis and theorycrafting is "pure." It doesn't take emotions, houserules, biases, etc., into account--it only cares about the system and the mechanics. There's great value to be had in that, because any playtest report is going to be muddy water at best.

Now, I'm not saying Playtest reports have no value--far from it. I just don't think they should be put on a pedestal above other kinds of analysis. They should be considered equally valuable, not the only feedback that is valuable.


1 person marked this as a favorite.

I would suspect that PF *wants* emotions included in the feedback.


4 people marked this as a favorite.

Thank you for this. I've been trying to find a way to express these sentiments but I doubt I'd ever be able to do so as clearly, understandably, and professionally as you have here.


1 person marked this as a favorite.
Pathfinder Adventure Path, Rulebook, Starfinder Adventure Path, Starfinder Roleplaying Game Subscriber

I'm far from being an expert, but my feel from lurking is that there are levels of analysis. An in-depth treatise considering all the facets will get respect and attention. Others will post things they describe as analysis ("The arcanist is too powerful because it has all the adaptability of a wizard combined with the single day flexibility of a sorcerer").

I'll agree that the former sometimes gets lumped in with the latter. That's unfortunate. The latter might be true in many cases, but it often lacks context of mitigating factors that will come up in play.

An example: it's not hard to imagine a new player coming to Pathfinder and looking at all the classes and coming to the conclusion that the rogue must be a damage master. I mean, look at all the dice! Comparing that to other classes, like the fighter, that only get plus a few points of damage (here I'm referring to weapon training and potentially weapon specialization) means they should dominate combat.

But in the context of the game, there are a number of mitigating factors. First, even though rogues use rapiers and other high-crit weapons, their bonus damage doesn't interact with those crits. Plus, fighters are able to focus on strength and power attack will make up a lot of that damage difference. While rogues have these options, power attack takes away to-hit bonus, which rogues struggle with and if they focus on strength to the exclusion of dex and such, they are far too squishy. These difficulties will come up in play, at least in aggregate.

What's the point of my example? Yes, analysis is pure while playtest data are biased. But playtest data tell the whole story in ways that are likely to get missed in analysis. This isn't a design session, it's quality assurance. Playing the class is where you find out if it actually feels right. Analysis also gives you the right ballpark, but playtesting gives you the fine-tuning (again in aggregate, not from individual responses).

Finally, it's not that they don't want analysis. It's just that they're manipulating us to get the ratio of response types to be more optimal. One good analysis with peer review gives a lot of great information. One hundred middling analyses are pretty worthless to them since they need to take as much time poring through them as it would take them to do it themselves. Likewise, one pure playtest is not worth much to them (for the very valid reasons you listed here). They can, and almost certainly do, do that themselves. A thousand playtests tell them what needs tweaking and what doesn't feel right.

So it's not that playtests are each more valuable information than analysis. It's that by encouraging us to playtest, they get the information they can't get internally. Plus, the people excited about analysis will still do it. But by limiting the number of full-bore analyses that happen, they (and the community) have time to prod those analyses and see what's true, what's missed, etc. rather than getting lost in a sea of conjecture.

TL;DR: Many shallow playtests+some deep analysis>>>>some deep playtests+many shallow analyses. They tell us what they think it takes to get what they want, not exactly what they want.


5 people marked this as a favorite.
Zhayne wrote:
I would suspect that PF *wants* emotions included in the feedback.

I wouldn't say that. Sean K. has been talking against emotional feedback of the negative variety rather a lot during this playtest.


Pathfinder Adventure Path, Rulebook, Starfinder Adventure Path, Starfinder Roleplaying Game Subscriber
Arae Garven wrote:
Zhayne wrote:
I would suspect that PF *wants* emotions included in the feedback.
I wouldn't say that. Sean K. has been talking against emotional feedback of the negative variety rather a lot during this playtest.

My most optimistic interpretation of what emotional response could mean, though, is does it feel like you're a swashbuckler when you play a swashbuckler?

With no disrespect to analysis (I lean that way myself), until I play a class, I'm too influenced by the written fluff to be sure if the crunch matches what I want it to.

Am I moving around as I fight or does that cost me too much? Does Parry model in-game what Inigo Montoya does when fighting the Man in Black? I can use analysis to figure out if it will work against dragons and such (which is valuable), but the sense of it isn't there without the context of a game. At least for me. That is what I think of as my emotional response to the classes.

RPG Superstar 2015 Top 8

9 people marked this as a favorite.

Look, to an extent I get your point, BUT, I think the reason the devs focus on experiential play feedback is because

1. There is stuff that can look "broken" on paper but then actually never pan out as a problem in play. This happens a LOT. I've seen a lot of useless, pointless houserules come out of a GM who anticipated a problem that didn't exist---houserules which often result in something else breaking---and the whole situation could have been avoided if the GM had just actually TRIED the rules as written before trying to fix something.

2. Likewise there is the reverse: there is stuff nobody notices as an issue when reading but turns out to be utterly game breaking in play.

3. Theorycraft tends to assume a certain ideal set of circumstances that seldom effectively accounts for game-affecting factors like environment, terrain, party makeup, party roles, and a variety of other idiosyncracies that you encounter in play but seldom remember to account for when theorizing.

4. Ultimately, knowing how a class FEELS when played than how it LOOKS is more important.

That said, I do wish the devs would pay a little more attention to some of the non playtest analysis because it does contain some good material. I have a feeling however that if they SAID they valued such analysis, it would make the already burdensome theorycraft blather dominate the forums even further and it would be that much harder to see how things work in practice.

Absolutely, there is value to statblock comparisons, number crunching, and simple reactions to things like "this doesn't look fun to play" (people buy books after flipping through them, they don't ask the store owner if they can sit down and start a game; players pick race and class based on what looks fun to them usually rather than what they've already tested out). But the fact is, gamers are going to offer that information even if you ask them not to, so it's probably not wise to push for it even more. At the same time, gamers may not get around to putting together a playtest session unless you really push for it though--after all, it is a MUCH, MUCH greater demand on what is entirely volunteer, uncredited time and contribution. Unfortunately if you're too busy to put together a playtest and therefore all you can contribute is your impressions, it can feel frustrating. But that unfortunately just is what it is.

Also, I admit I myself sometimes struggle to remember this is a beta, not an alpha. Certain design decisions are set in stone and certain decisions have been made and they just want to test out what they've established, not get wishlists of things they have not included. People often start getting off topic into things the designers are not in a place to address and don't wish to be, and reminding them this is about working with what they've been given is important to keep everyone on the ball.


5 people marked this as a favorite.
Berinor wrote:
Many shallow playtests+some deep analysis>>>>some deep playtests+many shallow analyses.
I think you are discounting "shallow" analyses. To be useful, an analysis of part of a class doesn't have to be a thousand words and include an appendix of an R program. For example, take this bit from mplindustries's post:
mplindustries wrote:
They'll see how fun it is [playing a swashbuckler] at low levels (well, levels after 1, since they suck for a whole level), and never get to the point where they're failing instant death saves left and right because they're the "front line muscle" but have only a good Reflex save.

It only took a single sentence to point out that the swashbuckler's save progression severely hampers their capability to perform their intended role. It didn't take deep analysis to get at this problem. Sure, this sort of theorycrafting doesn't give the developers any information they couldn't get internally. But that just raises a question: if the flaws noticed by this theorycrafting are so easy to catch, then why did they make it to the public playtest?


4 people marked this as a favorite.

As Sean K Reynolds so snarkily pointed out in his response to my comment on the other thread, I have not participated in playtesting much in the past. I did do some playtesting for WotC, but that doesn't really matter here.

He has a point, I would agree that my lack of actual play of these classes is something that needs to be weighed in, but as mplindustries points out above, theorycrafting and plain old fashioned analysis should not be discounted, especially just because the developers don't want to hear it.

My problem with play testing is simply time and opportunity. My play time is very limited, and what play time I do have is currently invested in two long-term ongoing campaigns, so playtesting just isn't going to be on my agenda.

I do agree that playtesting alone has some potential pitfalls that Paizo should take into account, and I think they will. The bigger issue to me isn't whether the feedback is coming from playtesters or not, but that at least this round the response to criticism has been somewhat sensitive and potentially counterproductive.

And again, I wish Paizo would take some of the time, energy and money invested in these new classes and fix some of the well-acknowledged problems with the existing ones.

But that's just me.

Scarab Sages

Playtesting is what they can't do that well in house?


feytharn wrote:
Playtesting is what they can't do that well in house?

I would find this difficult to understand since Paizo is pretty much the only company on earth that will actually pay people to play the game on company time.


There's nothing wrong with theory crafting, it's definitely useful. It's just that it needs to be tempered with actual play experience. It can't always pick up on the benefits of noncombat class features, utility abilities, or sometimes defensive abilities (I've noticed in the brawler vs monk analysis, high will saves seem to be dismissed for some reason). Some abilities may be useful, but only in certain circumstances which may come up more or less frequently in different groups. Theory crafting also typically assumes very high levels of optimization, which is certainly not the case for a significant portion of the Pathfinder fan base. How does the class work in casual settings?

The biggest reason, however, is that theory crafting can't answer one extremely important aspect of he class: is it fun to play?


13 people marked this as a favorite.

Why is experimental science considered more important than natural philosophy?


2 people marked this as a favorite.

To me, it looks like the OP here assumes that the very presence of the playtest somehow rules out any theorycraft or statistical analysis going on behind the scenes, or somehow lessens its importance to the game designers.

As with all assumptions, it's likely false. The game is stacked to the nines with stats and numbers. It can be fairly (and better) assumed that they care plenty about this.

The simplest answer is that playtesting is one aspect of game design that the designers can get the whole community involved in, so they do. If it seems like a big deal, that is because hundreds of us respond and argue to make it so. This is no reflection on the designers whatsoever.

In short, playtests keep the community involved and interested, like any other marketing tool. They are not the end-all-be-all of game design, and any theory that they are so is wild, baseless assumption that flies in the face of the game's very existence and presentation.


3 people marked this as a favorite.
Cheapy wrote:
Why is experimental science considered more important than natural philosophy?

One might argue that theorycrafting hits a wee bit closer to the mark than say, platonism.


I think the reason there's so much emphasis on playtesting is because there's going to be people posting flat opinions/observations regardless. By constantly asking people to playtest, I think they're just trying to swing the percentages a bit more evenly.

Shadow Lodge

I believe one of the main reasons, as the developers have said on more than one occassion is that when creating new rules a lot of what they are doing is theorycrafting. If they wanted opinions based only on theories based on what it looks like they can do it themselves. Playtesting is really a way to get the community involved and make them feel like its more their game than just the developers. Also, just looking at how it balances up based on theory isn't as effective as playing an arcanist alongside a wizard and seeing if they have the same basic power level or if the new class is supremely more or less powerful than the old.


4 people marked this as a favorite.

Paizo gets a few gems of wisdom from playtests, a few tweaks that need tweaking, I'm sure, but I'd wager the most valuable outcome of playtesting is player ownership. Let the player base - the clientele - take the new toy for a spin so that, when the time comes to purchase that toy, they already believe it is theirs. Purchasing is a forgone conclusion because we've already experienced owning it and we can't very well LOSE something we already had. No company is more open to their customer base than Paizo, and I love them dearly for it, but I believe that playtesting is 90% a business decision.

(No criticism intended. I do love me a good test-drive. And I hope Paizo's smart business decisions keep many more playtests coming in the future.)

Dark Archive

DeathQuaker wrote:


That said, I do wish the devs would pay a little more attention to some of the non playtest analysis because it does contain some good material. I have a feeling however that if they SAID they valued such analysis, it would make the already burdensome theorycraft blather dominate the forums even further and it would be that much harder to see how things work in practice.

This, and also, a lot of the theorycraft is off-base still. Sturgeon's Law very much applies to message boards and it's a lot easier for them to encourage playtest rather than wade through piles of conflicting theorycraft. After all, if they make it clear how much they value playtesting, they might convince people to playtest that might otherwise not have done so. This isn't a knock against theorycrafters, just an attempt to focus the feedback.

Adamantine Dragon wrote:
feytharn wrote:
Playtesting is what they can't do that well in house?
I would find this difficult to understand since Paizo is pretty much the only company on earth that will actually pay people to play the game on company time.

I doubt the entire company can be dedicated to playtesting 100% (including the artists and lawyers for instance) and even if they could, it would still be a paltry number compared to the forums.

Liberty's Edge

8 people marked this as a favorite.

Because most people play the game, not theory craft the game.


6 people marked this as a favorite.

While I don't disagree, I think the problem is that Paizo has 95% analysis right now and 5% actual playtest feedback.

Both are important, no one is arguing that. But they need people to actually play the classes to find out how the mechanics work live.

It's the same as any other experiment. Analysis can only go so far.

Sean, Jason and the others have a LOT of pages of analysis. What they also need is playtesting to round that out.

So let's get some games going!

Liberty's Edge

1 person marked this as a favorite.
Cheapy wrote:
Why is experimental science considered more important than natural philosophy?

More to the point, prior to testing people believed heavier objects fall faster than light ones.

Adjusting for air, they don't.

Theory crafting is all well and good, but it cherry picks variables. It isn't real world application in actual game setting.

It has it's place, but I would hope before Boeing puts a plane on the market they test it rather than just go by the models.

Cause the batteries didn't catch fire in the models...


1 person marked this as a favorite.

For me the problem of playtesting, in ths case, is that there are so may calss that I just not care about.

I do not really want to play them. Brawler are bland, swashbuclker have too many features from the fighter and duelist, warpriest can already be done with multiclassing, skald...thanks but no. They will not take this into consideration, and that is a shame.

Their mechanic can be solid (or not) but they are just uninteresting to me, too much is recycled.

In case somebody cares I will join a playstest PbP, probalby with a shaman since we have already an slayer and an investigator (the only two other classes that I do really want to try).


5 people marked this as a favorite.

Nicos, what you are describing above is what actual experimental scientists call "selection bias" and it's an important thing to understand. People tend to play test what they like, and since they like it, they rate it highly.

In game playtesting this isn't necessarily bad if two things are true:

1. There are enough play testers with a wide enough interest base that all the options are adequately tested.
2. The population of play testers accurately reflects in general tastes and interest, the population of game players.

Both of those may be true. I would assume that Paizo is keeping some sort of statistical tracking of how much testing each class is getting. whether the play testers accurately reflect the potential gamer base is another question entirely.

One other comment. The value of theorycrafting is dependent on the theorycrafter. Some people have an amazing capacity for theorycrafting remarkably well. Some of those people are on these boards.

Liberty's Edge

@Nicos - And that is good playtest feedback actually.

I don't want to play this class is good information.

Of the 10, I want to play the Bloodrager, Brawler, Investigator, Slayer, and Warpriest.

I can see my friends wanting to play the Arcanist and Shaman.

I have no interest in playing the Skald, Swashbuckler or Hunter.

Does that mean I hate half the classes? No, it means I love 5, 2 I can see my friends loving and 3 aren't ones I expect to be in by group.

If we get a chance to playtest, I can then see how the class "feels" at the table. Does the mechanic match the flavor? Is it confusing or clunky to run? Am I over powered? Under powered? Is it boring.

Theorycrafting gives none of this. Particularly what generally passes for conversation on here where builds aren't even involved, meaning each side can cherry pick variables...

What is always comes down to is "Will my group enjoy playing this class, and therefore buy lots of Modules and Adventure Paths so we can do that more often?"

And theorycraft provides no answer to that question.


2 people marked this as a favorite.
Psyren wrote:
This, and also, a lot of the theorycraft is off-base still. Sturgeon's Law very much applies to message boards and it's a lot easier for them to encourage playtest rather than wade through piles of conflicting theorycraft. After all, if they make it clear how much they value playtesting, they might convince people to playtest that might otherwise not have done so. This isn't a knock against theorycrafters, just an attempt to focus the feedback.

I would understand this point of view if the theorycrafting was actually conflicting, but in general, it's fairly unified on several issues.

Hunters, Warpriests, and Skalds, for example, have legitimate problems that the vast majority of analysts agree on. Swashbucklers have a lot of conflict about weapon choice, but the save issue is again, acknowledged universally. But these obvious mechanical issues keep getting swept under the, "I can't wait to hear about your playtest" rug.

In fact, other than minor typos and clarifications, the only issue that has really been addressed so far in the playtest was non-mechanical. Just about everyone complained about the (lack of) flavor in the Arcanist and they swiftly went about correcting it. This bothers me--they're putting "feel" above effectiveness.


The same reason science> than philosophy.

Silver Crusade

7 people marked this as a favorite.
Pathfinder Adventure Path Subscriber

Theorycraft doesn't involve emotions? What about all those 'Paizo didn't listen to my sagely advice, so I'm out to tear them a new one?' people?

Liberty's Edge

1 person marked this as a favorite.

I love how things people personally agree with are then "universally acknowledged"...

Why do we even bother playing the game, anyway?


Gorbacz wrote:
Theorycraft doesn't involve emotions? What about all those 'Paizo didn't listen to my sagely advice, so I'm out to tear them a new one?' people?

They turn into bags of devouring :)


5 people marked this as a favorite.

I will be interested in how well a new book full of new classes sells. From my own perspective there are other things I'd rather see done with the skills and passion of the Paizo developers than contrive new classes that fill pretty much the same roles as existing classes. In my own game play, and with the people I play with, I don't hear a lot of "you know what this game needs? Another melee based arcane caster class." What I hear a lot of is "I wish they'd fix the monk" or "I want to play a rogue, but can't understand why they suck at scouting" stuff like that. I wonder what sort of feedback Paizo has been receiving from other sources since that's the sort of stuff that dominates these boards too.

It is hard to come to any other conclusion than "well new splat books provide income."

Which is why I really am interested in how well this sells. I frankly doubt a single player in my group will buy it. But they'd definitely pay for an update that fixes the known problems they complain about regularly.

Liberty's Edge

@AD - And they gave monk and rogue alternative classes...


1 person marked this as a favorite.
Jessie Scott wrote:

While I don't disagree, I think the problem is that Paizo has 95% analysis right now and 5% actual playtest feedback.

Both are important, no one is arguing that. But they need people to actually play the classes to find out how the mechanics work live.

It's the same as any other experiment. Analysis can only go so far.

Sean, Jason and the others have a LOT of pages of analysis. What they also need is playtesting to round that out.

So let's get some games going!

This so much.

We already have the mounds upon mounds of theorycrafting.

Paizo has probably already came to mostly the same conclusions that we had when they released the playtest. Its hard to imagine that they wouldnt think, "Hrm, a Swashbuckler with only 1 good save is going to have a hard time at high levels." Somehow people still play rogues at those levels so why not? If enough test data proves otherwise, they may change their mind.

Obviously we have some goals in mind for this playtest:

Balance
Concept/Fluff
Effectiveness

Personally when I get around to my 10th level playtest, I'm going to build a gauntlet of different encounters and see how the folks run.


3 people marked this as a favorite.
ciretose wrote:
I don't want to play this class is good information.

I look forward to hearing about your playtest to prove it. ;)

ciretose wrote:

Of the 10, I want to play the Bloodrager, Brawler, Investigator, Slayer, and Warpriest.

I can see my friends wanting to play the Arcanist and Shaman.

I have no interest in playing the Skald, Swashbuckler or Hunter.

Does that mean I hate half the classes? No, it means I love 5, 2 I can see my friends loving and 3 aren't ones I expect to be in by group.

I don't find myself especially interested in playing any of these classes. I'd consider the Investigator, maybe. But its unlikely. I can see plenty of people I game with wanting to be an Investigator or Slayer.

If things don't change, however, I can also see having to constantly counsel players away from Hunters and Warpriests, pointing out that they can get exactly the feel they want through much better sources.

So, since I don't really want to play as any of these classes, does that mean my feedback is pointless to you? I don't need to do to understand. I can see what a lego house will look like in my mind without building one first.

ciretose wrote:
If we get a chance to playtest, I can then see how the class "feels" at the table. Does the mechanic match the flavor? Is it confusing or clunky to run? Am I over powered? Under powered? Is it boring.

And I see the value in playtesting for the purposes of a few of those questions (the feel, mechanic matching flavor, confusion/clunkiness), but the playtest will make questions of power meaningless. There are too many variables in live play, especially houserules, GM styles, GM intelligence/tactics, etc.

You need outside analysis to judge power. Any other discussion of power is biased by the table.


1 person marked this as a favorite.
ciretose wrote:
@AD - And they gave monk and rogue alternative classes...

ciretose, I find you to be one of the most rational and valuable commenters on these boards, so your comments have heavy weight with me.

But is it your true opinion that the way to "fix" the rogue and monk is to replace them with new classes that might address the old classes' problems but also change the flavor rather dramatically?

I've got a player who loves monks. He has been bummed out since we converted to PF by his inability to create a monk that satisfies his goals. He's not going to want to play a replacement class. He wants to play a monk.

Liberty's Edge

1 person marked this as a favorite.
Adamantine Dragon wrote:
ciretose wrote:
@AD - And they gave monk and rogue alternative classes...

ciretose, I find you to be one of the most rational and valuable commenters on these boards, so your comments have heavy weight with me.

But is it your true opinion that the way to "fix" the rogue and monk is to replace them with new classes that might address the old classes' problems but also change the flavor rather dramatically?

I've got a player who loves monks. He has been bummed out since we converted to PF by his inability to create a monk that satisfies his goals. He's not going to want to play a replacement class. He wants to play a monk.

I don't think they are going to be able to "fix" the monk or rogue until the next edition. Doing so would obsolete the core version, and that isn't something I see them being interested in doing.

I think "a" monk is pretty good right now. It just isn't the unarmed monk.

I think the investigator or slayer fill the two things people complained about wanting in a rogue.

While neither is the perfect, I try to avoid having the perfect be the enemy of improvement.


Unless, of course, we have tables enough. I'm not sure there will be enough over a month, though.


1 person marked this as a favorite.

Before I start I will say that "actually" testing anything normally gives the best results. I can write a program that is supposed to make Unix and Windows 8 work perfectly together, but I may find out that in actual use there were things I did not consider or maybe I actually added to many features .

It seems that you(OP) made arguments for both sides except table variation. Table variation is good for the playtest because the idea is to provide classes that can work for mostly everyone. If everyone played the seem the playtesting would not be needed as much , if at all.
As an example I never had a problem with the original smite for the paladin, but enough people complained about it, that it got fixed, even though it was after the playtest. That is an example of how the community use something can be beneficial across the board. By seeing how the class actually works we can come up with actual solutions.


mplindustries wrote:

And I see the value in playtesting for the purposes of a few of those questions (the feel, mechanic matching flavor, confusion/clunkiness), but the playtest will make questions of power meaningless. There are too many variables in live play, especially houserules, GM styles, GM intelligence/tactics, etc.

You need outside analysis to judge power. Any other discussion of power is biased by the table.

Ideally any playtesters will list any houserules used. I know everyone won't but I think most would. I intend to do my playtest without any houserules so as to not skew data. GM play styles can be picked up on if the does a good description of the combat, and a more descriptive combat will be more likely to have more influence than someone with a "not so descriptive" combat/encounter.

edit:I have picked up on rules being broken from post on the messageboards before so it can be done.


Also, there's almost no limit to the amount of theorycrafting they can do in house. There IS a limit to the amount of playtesting they can do (unless they want to arrange all of your shipments into flat rolling surfaces and keep them there for a few weeks and see how things go)


3 people marked this as a favorite.

Rogue and Monk are often the subject of a lot of arguments on this board.

Its almost unanimous that Rogue is a weak class.

However I've never had a campaign without a Rogue. People like playing Rogues which may be more important to Paizo than how strong the Rogue actually is.

I'd agree with the original point if there was way more playtest feedback than theory-crafting, but as it stands now;

1) I've been following this on the message boards since it was announced and some people have been theory crafting before the pdf even came out. Practically within 5 minutes of the pdf hitting the net 'reviews' started popping up.

2) I think some changes have already happened due to what people have pointed out without playtesting so I don't think they're ignoring what's being said, they just value playtest feedback more due to it's scarcity.

3) In some cases playtests are really needed. I have not played or seen a Brawler yet and I really need to make one before I can form a real opinion on it's Martial Maneuvers class feature. People like it, and it looks good on paper but it makes me nervous of people looking through many books to find the right feat for the job. But Players could just be lazy and make a list of a few feats they like and stick with that, then I have nothing to worry about.

4) I've kept my mouth shut about my builds because I have not played them but some things that are complained about are not my complaints, at least on the character sheets.


1 person marked this as a favorite.

Theorycraft is great for trying to figure out potential. But poor in application. It's trying to determine how things work in a sterile lab environment.

Playtesting figures out how things work in the wild. Given that the product itself is going to see a lot more use in the wild being actually played rather than poured over this is ultimately more useful to designers.

The points about things seeming overpowered without being played has a point. The Mystic Theurge was considered overpowered....right up until you played it. This process gets repeated over and over again with each new class or serious set of options that gets released.

And ebar in mind Playtesting isn't jsut sitting around in a group and playing it. IT can include things like what Rogue Eidolon did with the swashbuckler. Things I've done like building test groups and running them through encounters (in which case I'm player and GM and look for interactions rather than spreadsheets).

For my part I'll end up buying it. I love new player options because I can cram those in my GM space and expand that box as well. When the players get the crap kicked out of them by a bunch of naked orc brawlers using Boar style feats it will be hilarious.

Dark Archive

mplindustries wrote:


In fact, other than minor typos and clarifications, the only issue that has really been addressed so far in the playtest was non-mechanical. Just about everyone complained about the (lack of) flavor in the Arcanist and they swiftly went about correcting it. This bothers me--they're putting "feel" above effectiveness.

As DeathQuaker told you in the beginning, this is a beta, not an alpha. Fluff is something they can easily change, mechanics not so much.

And yes, "feel" is very important. Effectiveness is going to vary wildly from playgroup to playgroup. I have, right now at one of my tables, someone inexeperienced playing a wizard and getting outperformed in every encounter by the dancing bard - yet every time he gets to grab the pyramid d4s for magic missile he's having the time of his life.


1 person marked this as a favorite.
ciretose wrote:
Adamantine Dragon wrote:
ciretose wrote:
@AD - And they gave monk and rogue alternative classes...

ciretose, I find you to be one of the most rational and valuable commenters on these boards, so your comments have heavy weight with me.

But is it your true opinion that the way to "fix" the rogue and monk is to replace them with new classes that might address the old classes' problems but also change the flavor rather dramatically?

I've got a player who loves monks. He has been bummed out since we converted to PF by his inability to create a monk that satisfies his goals. He's not going to want to play a replacement class. He wants to play a monk.

I don't think they are going to be able to "fix" the monk or rogue until the next edition. Doing so would obsolete the core version, and that isn't something I see them being interested in doing.

I think "a" monk is pretty good right now. It just isn't the unarmed monk.

I think the investigator or slayer fill the two things people complained about wanting in a rogue.

While neither is the perfect, I try to avoid having the perfect be the enemy of improvement.

Ah, so does that imply that you have more or less given up on the existing issues in the already published rules then?

Last I checked the Paizo staff on these boards have said they have no current plans to do a 2nd edition. So to say they won't be fixed until a new edition essentially is the same as saying they won't be fixed.

Which I think is a problem that can't be fixed with contrived new classes that will just cost people more money and which GMs may or may not accept in their games.

I have to admit that "Oh you want a fix for the monk? Buy this new book and accept the new flavor, oh and talk your GM into accepting (and buying) it too" isn't really the sort of "solution" to the problem that I would prefer.

I can see it bringing smiles to the finance dept at Paizo though.


1 person marked this as a favorite.

I believe playtesting can reveal things that aren't readily apparent to most. However, I don't think you need playtesting to analyze that the hunter is weaker than the druid and that the warpriest is weaker than almost anything you could cobble together from what is currently available from divine classes. While I don't believe all of the classes have to be balanced, there should be some advantage to playing a class. I believe there is a fairly large amount of "optimizers" that would never play those two classes in their current form. In that respect, you might not get negative playtest feedback on those classes because the "optimizers" won't playtest them.

Playtesting does have some value. My friend converted a monk in our shattered star campaign to a brawler and was disappointed with the martial maneuvers ability, because it required a move or standard action to get more than one feat. When I look at the brawler, my theorycrafting analysis is that without a better martial maneuver ability, it doesn't give me a reason to play that class over a monk. Monks already have full bab while flurrying. Brawler loses ki pool and possible qigong powers. What the brawler gains mechanically isn't worth what it loses, so the only way I would play it is if the martial maneuvers ability was good enough to make up for it.


Adamantine Dragon wrote:
ciretose wrote:
Adamantine Dragon wrote:
ciretose wrote:
@AD - And they gave monk and rogue alternative classes...

ciretose, I find you to be one of the most rational and valuable commenters on these boards, so your comments have heavy weight with me.

But is it your true opinion that the way to "fix" the rogue and monk is to replace them with new classes that might address the old classes' problems but also change the flavor rather dramatically?

I've got a player who loves monks. He has been bummed out since we converted to PF by his inability to create a monk that satisfies his goals. He's not going to want to play a replacement class. He wants to play a monk.

I don't think they are going to be able to "fix" the monk or rogue until the next edition. Doing so would obsolete the core version, and that isn't something I see them being interested in doing.

I think "a" monk is pretty good right now. It just isn't the unarmed monk.

I think the investigator or slayer fill the two things people complained about wanting in a rogue.

While neither is the perfect, I try to avoid having the perfect be the enemy of improvement.

Ah, so does that imply that you have more or less given up on the existing issues in the already published rules then?

Last I checked the Paizo staff on these boards have said they have no current plans to do a 2nd edition. So to say they won't be fixed until a new edition essentially is the same as saying they won't be fixed.

Which I think is a problem that can't be fixed with contrived new classes that will just cost people more money and which GMs may or may not accept in their games.

I have to admit that "Oh you want a fix for the monk? Buy this new book and accept the new flavor, oh and talk your GM into accepting (and buying) it too" isn't really the sort of "solution" to the problem that I would prefer.

I can see it bringing smiles to the finance dept at Paizo though.

So the alternative would be what exactly?

Liberty's Edge

@AD - I think the armed monk is pretty good currently, based on all the math. Being able to TWF with one weapon is...well...good.

The unarmed monk...not so much. I made my suggestions in the past, but what I've seen is basically they don't want either a band-aid or a re-write so here we are. It is usable but not optimal, and they are putting out the brawler (in addition to the martial artist archetype and all the style feats) for people who want that style and it is, by the math, now ok.

As to the rogue....steak attack is the core of the class and that is what it is. I have some suggestions, but frankly short of major revisions I don't see how you address the complaints people have for the people who are complaining.

So...yeah. This is a 13 going on 14 year old chassis. The bumps they gave to other classes worked better than the bump for the rogue, because sneak attack damage isn't as impressive now that all other damage has gone up significantly.

And I don't see them giving it a d8 as errata :)

RPG Superstar 2012 Top 32

i think there is a need for both (theorycraft and playtesting). the nice about theorycraft is that it can confirm/support the playtest experience, or help set practical expectations and/or clarify the underlying probabilities when two people have very different experiences with the same class.

the great thing about a playtest is that you see things that you almost certainly wouldn't think to put into a theorycraft analysis (or wouldn't bother unless you were doing a much deeper analysis than i've ever seen posted here). for example (from a playtest session with a fighter, a monk, and a brawler)- when faced with an enemy using spring attack, the fighter readies a vital strike to make the enemy think twice about that tactic... what does the brawler do (in this case, copy the fighter- spend a move to pick up Vital Strike and 2hand your temple sword); when you charge the eldritch knight and hit him soundly, rather than standing toe-to-toe he casts fly on the defensive and moves away to casting range- what now (in the test, he'd thought to buy a +1 mighty bow- so he used MM to pick up proficiency and counted on his pretty solid saves to last the longer-than-normal combat... though if the EK had some SoS will spells that might have gone less well).

also, as has been mentioned already, it gives you a better chance to see how it functions with other classes... (is the brawlers slightly lower AC and, potentially, HP as big a deal as some make it, or does even a secondary healer have the resources to keep him up through a fight?- you could theorycraft that, but most wouldn't/haven't and the playtest should give you pretty decent experiential evidence of it. and that's just one example- you'd have to set up hundreds of theorycraft scenarios to analyze everything you naturally gain some evidence about in a playtest).


1 person marked this as a favorite.

I'm repeating some things that have said (and adding my support to anything I'm repeating):

1) Playtesting shows what happens when a large number of gamers play the classes. An optimized specialist wizard will be more powerful than a lot of sorcerer builds, but a sorcerer is fairly easy to play and is still effective most or all of the time. The swashbuckler's saves are an issue on paper, but it might work differently in gameplay- a swashbuckler might be so useful at high levels the casters might always cast save-boosting buffs, or it could be a swashbuckler usually ends up multiclassing to boost saves, or the saves could be worse than they look on paper.

2) The Paizo developers are very good at theorycrafting, and don't have enough time to playtest each class from levels 1-20 before the summer. A fair number of theorycrafters will notice things Paizo devs won't and those theorycrafters will post without encouragement. And playtesters will catch things Paizo devs won't (sometimes a new player will stumble on a combo an experienced player won't think of, and it might be a powerful combo or a really weak combo).

3) People who design and write rpgs can easily miss something they wrote that needs to be fixed. In the first printed edition of Godlike, a character with a strength of one always drowns in water. You need two dice to attempt a swim roll. The designers intended for a minimum of two dice for any check, but didn't include that in the rules. So designers testing the system used the minimum of two dice for a check without including that in the book.

4) Extensive open playtesting will show how a lot of class combinations work. I'm guessing ACG will have upgrades for rogue and monk (and new options for all the classes), but it may be that a combo of skald, bard, rogue and monk can use cool combos. Or maybe skald, hunter, summoner, and druid can do 1000 points of damage in a round at fifth level. It could be a shaman ability effectively cancels out the benefit of another class feature that makes one or both of them waste a turn accidentally.


1 person marked this as a favorite.

@ciretose, you are probably right and Paizo will not be fixing any of the serious problems in the rules today. And rogue and monk are only two of the most common. Instead they will continue to stack more content on the already flawed foundation and hope it doesn't collapse under its weight as 3.5 did.

I think it's a risky bet, especially in the environment where their major competitor IS trying to deal with the fundamental flaws they are ignoring. As I've told others before who claim to be PF fans forever, the majority of them used to be TSR fans too, and you see where that got TSR.

I don't like the strategic direction that Paizo is pursuing. They had an opportunity to fix the flaws in the foundation but they chose not to. I guess whining about that is pointless and splat books with bandaids on major problems is the best we'll see. Maybe that will be enough to keep Paizo on the top of the market. We'll see.

@Tark - the alternative is as I have said all along, and as I have said for years now on these boards. Focus on fixing the broken stuff before piling more on top of it.

Clearly Paizo has decided that is not the most profitable approach to things.

Perhaps they are right. I suspect they are just perpetuating the cyclical edition problem and PF will become yet another version with its own set of grognards as the game moves on. Time will tell.

1 to 50 of 167 << first < prev | 1 | 2 | 3 | 4 | next > last >>
Community / Forums / Archive / Pathfinder / Playtests & Prerelease Discussions / Advanced Class Guide Playtest / General Discussion / Why is playtesting considered more important than other forms of analysis? All Messageboards