Why not poll the community first?

[Verzen]

just a thought... before creating new classes, wouldnt it save a lot of time to first get community feedback and run a poll on what we want to see as a community before releasing playtest versions that get utterly annihilated then brought back to the drawing board and the new concept for the class is never playtested?

I'm worried about both magus and summoner now...

[Martialmasters]

It won't amount to much if they did. The community can be a good measure of what's wrong. But almost never a measure of the correct direction to go.

[Salamileg]

There were a lot of threads about the Magus and Summoner prior to the release of the playtest.

People had so many different ideas on what they wanted the classes to be that I don't think it would have actually been helpful. Releasing a playtest, even if people don't like it, gives everyone the same frame of reference to compare ideas to.

[Salamileg]

Martialmasters wrote:

It won't amount to much if they did. The community can be a good measure of what's wrong. But almost never a measure of the correct direction to go.

This is also pretty true. Most people disliked the playtest Investigator, but I don't think anyone suggested Devise a Stratagem, which is a fantastic mechanic (coincidentally, I believe the same dev that's leading the Magus made the Investigator, so I have high hopes).

[Capn Cupcake]

Yep. There's a saying in retail that applies here. "People never know what they want, but they always know what they don't want."

[Voss]

Yeah. Design needs a direction first, feedback later. 'Feedback' first means getting lost in the desert really fast.

[Tender Tendrils]

Never design by public committee. You can get feedback from the public after they play the thing, but don't ask them beforehand as they generally think that they want things that in practice they wouldn't like.

[Ascalaphus]

Verzen wrote:

just a thought... before creating new classes, wouldnt it save a lot of time to first get community feedback and run a poll on what we want to see as a community

People have been making threads like that on the forum for the past year. I see a lot of the issues and ideas from those threads coming back in this playtest actually, like the magus having better weapon training than the war priest, or the summoner trying to have a pet that's more powerful than a druid's but balancing it somehow.

Verzen wrote:

before releasing playtest versions that get utterly annihilated

If it was bad it should be annihilated.

If there are parts of the playtest you really like, tell the designers in the surveys, and point it out on the forum so that other people may also report it in surveys, and there's a good chance it stays.

Verzen wrote:

then brought back to the drawing board and the new concept for the class is never playtested?

Here I somewhat agree with you: these are single-iteration playtests. If you have all the time and money in the world you'd like to do multiple iterations and zero in on things. However, as was shown in the main 2E playtest, going multiple rounds also tires out your playtesters (and designers).

It's pretty common actually that when designing a new product of any kind, you can only afford so many rounds of experiments, because the experiment itself is expensive to carry out. A car manufacturer that has to do crash tests has to fabricate a new car for every test, so you want to do as few experiments as is needed. If you can get by with some simulations first to weed out the designs that are deeply flawed, then you only have to do experiments with potentially final designs, or if you need to test out a new technique that's so different that you don't know how to simulate it well.

That's also what's happening here. Running a playtest isn't cheap - you have to write up a playtest document, set up surveys, get people hyped to play the playtest and report back within a timeframe that works for you, then actually process all the feedback you got on forums and surveys. And if you do a second round of playtesting, you have to get people hyped again and get them doing surveys again, again in a timely manner. That's a lot of work. So you want to get the maximum information for each round of playtesting you do.

This is also why these playtests tend to be so wacky, with weird off the wall mechanics. Safe, well-understood mechanics can be tested in-house which is much faster and cheaper. It's the crazy out there ideas that need outside testing.

In setting up experiments, if you're not ending up rejecting a substantial portion of the designs afterwards, your experiments are probably not daring enough to yield a high amount of insight per experiment.

[Temperans]

The problem with just throwing wild experiment is that you need a control to understand where things break down. This is best seen in the APG playtest where each class had an entirely different response. And even thou things like Oracle and Witch got a lot of negative comments they still kept parts of the bad design.

The problem being that people who will like whatever Paizo releases will just continue to like those things even if its entirely experimental. Paizo then gets mixed results and ends up with weird data. Specially when just like you said creating a playtest takes time.

So even if internal testing is cheaper, there is a high risk that the version we receive via the playtest is close to their final design. Because redesigning a class from 0 after it gets bombed is incredibly time consuming and wont have any feedback on what to fix.

So yes playtests need to have some amount of experimental stuff. But there is a limit to how experimental options can be before the data becomes a useless waste of time.

*******************

Like all the attempts at resonance that just kept failing and resulted in a horrible initial release of Alchemist.

[Lightning Raven]

Even though it seems like a good a idea, it ultimately isn't because creators that cater too much to their fans/base always end up with things that don't please anyone while trying to please everyone.

I much rather have strong reactions of likes or dislikes than just "meh".

On the other had, some vague polls before tackling a new class wouldn't be so bad, just to see what people think a given class is all about or which mechanical aspect is perceived as the most representative. That could produce guidelines that are loose enough for creativity.

[Lelomenia]

Honestly, playtest Magus is pretty much exactly the direction the community has indicated it was looking for. ‘Good martial, good caster, that it is built around a class feature that gives special bonuses for Magi when they combine weapon+spell attacks together. With good saves! And different special magic stylestances for combat!’

It’s just that the Striking Spell bonus effect isn’t something a lot of people want (crit fishing spikiness).

Make the slide effect the built-in bonus for Magus Striking Spell and the crit bonus a synthesis and i think 90% of people would be fine with everything.

Oh, and we need a cantrip or focus spell that you don’t Feel Bad for Striking Spell with. Right now all the cantrips are worse than Electric Arc with Striking Spell, and Electric Arc Striking Spell is much worse than two-target Electric Arc.

[Unicore]

Lelomenia wrote:

Honestly, playtest Magus is pretty much exactly the direction the community has indicated it was looking for. ‘Good martial, good caster, that it is built around a class feature that gives special bonuses for Magi when they combine weapon+spell attacks together. With good saves! And different special magic stylestances for combat!’

It’s just that the Striking Spell bonus effect isn’t something a lot of people want (crit fishing spikiness).

Make the slide effect the built-in bonus for Magus Striking Spell and the crit bonus a synthesis and i think 90% of people would be fine with everything.

Oh, and we need a cantrip or focus spell that you don’t Feel Bad for Striking Spell with. Right now all the cantrips are worse than Electric Arc with Striking Spell, and Electric Arc Striking Spell is much worse than two-target Electric Arc.

DO ranged magus really want the slide synthesis built in and the crit rider as a choice that has to compete with being able to spell strike at range?

Also no math I have seen shows that electric arc is better than telekinetic strike for spell strike. In play, I think some people like the security of doing damage on the miss, but by the numbers the overall damage output is lower. Produce flame is also pretty awesome if you can get flanking on your weapon attack.

[gnoams]

I recall reading a post by paizo designers before that said something along the lines of we are making what we want to make, not whatever is most popular demand.

[Lelomenia]

Unicore wrote:

DO ranged magus really want the slide synthesis built in and the crit rider as a choice that has to compete with being able to spell strike at range?

Also no math I have seen shows that electric arc is better than telekinetic strike for spell strike. In play, I think some people like the security of doing damage on the miss, but by the numbers the overall damage output is lower. Produce flame is also pretty awesome if you can get flanking on your weapon attack.

ranged characters like to move too. Agree it’s not particularly optimal for them, but then neither is the flanking-hungry crit version.

Telekinetic gets more benefit from the Crit rider than Arc, but Arc was usually better without the rider in my numbers. Telekinetic eventually wins out if you get a high enough % of your hits to be crits, but (1) ‘eventually’ was sometimes over 50% crit rider (2) electric arc could switch off with chill touch to attack a lower save and the math is the same (3) Telekinetic becomes a MAP problem if it doesnt fire off in round 1, so even in scenarios where it slightly outdamages Arc with 0 MAP it falls behind in practice.

[The Gleeful Grognard]

Capn Cupcake wrote:

Yep. There's a saying in retail that applies here. "People never know what they want, but they always know what they don't want."

People often think they know what they don't want.

I have had many players who have stated:
- they don't like character death and harsher campaign themes
- they don't like travel tracking and survival mechanics
- they don't like 5e
- they don't like PF2e

All ask to come back to my games and give praise to games where their disliked elements were featured heavily.

People are fickle.

[PossibleCabbage]

I'm cool with the people who have considerable experience designing for this system doing the first design pass.

[Temperans]

The Gleeful Grognard wrote:

Capn Cupcake wrote:

Yep. There's a saying in retail that applies here. "People never know what they want, but they always know what they don't want."

People often think they know what they don't want.

I have had many players who have stated:
- they don't like character death and harsher campaign themes
- they don't like travel tracking and survival mechanics
- they don't like 5e
- they don't like PF2e

All ask to come back to my games and give praise to games where their disliked elements were featured heavily.

People are fickle.

Isnt that perhaps that you run good games and people want to play with you? Even if they dislike the systems and elements used.

The difference between disliking a movie trope but liking how a certain director uses it.

[Tender Tendrils]

Temperans wrote:

The Gleeful Grognard wrote:

Capn Cupcake wrote:

Yep. There's a saying in retail that applies here. "People never know what they want, but they always know what they don't want."

People often think they know what they don't want.

I have had many players who have stated:
- they don't like character death and harsher campaign themes
- they don't like travel tracking and survival mechanics
- they don't like 5e
- they don't like PF2e

All ask to come back to my games and give praise to games where their disliked elements were featured heavily.

People are fickle.

Isnt that perhaps that you run good games and people want to play with you? Even if they dislike the systems and elements used.

The difference between disliking a movie trope but liking how a certain director uses it.

If you run good games you probably use good systems to run those games, either because those systems help make them good games, or because having the knowledge and skill to run good games leads to you having the knowledge to choose good systems.

[Temperans]

Or you just know how to mitigate the problems of bad systems and make them palatable.

[The Gleeful Grognard]

Temperans wrote:

Isnt that perhaps that you run good games and people want to play with you? Even if they dislike the systems and elements used.

The difference between disliking a movie trope but liking how a certain director uses it.

I have also had people come back saying they have entirely changed their mind post my running certain mechanics (especially wilderness, tracking, travel and survival)

Sure, it is dependent on a GM doing a good job. But that applies to everything, if people enjoy it when it is run well then it doesn't change the fact that people enjoy it.

[WatersLethe]

Designing by poll would be such a s+%$ show.

Even when asking specific questions about specific mechanics that we personally experience in a playtest-class framework, it can be nearly impossible to draw decisive conclusions.

A cold poll filled with responses from armchair game designers with no context of different interactions and balance and design philosophy would be basically useless. It'd be able to generate some general sense of playerbase desires, but that's the REALLY easy part. That's day one class design meeting whiteboard stuff. Stuff they already know.

[Squeakmaan]

Cold polls would be pretty much useless, as found in any design process, most people don't want the things they claim to want or have any real idea how to accomplish the things they actually want.

[Megistone]

I imagine a rock band deciding to make a cover of a famous song, and asking their fanbase about how they should make it.
It's... not going to work.

[Midnightoker]

Salamileg wrote:

Martialmasters wrote:

It won't amount to much if they did. The community can be a good measure of what's wrong. But almost never a measure of the correct direction to go.

This is also pretty true. Most people disliked the playtest Investigator, but I don't think anyone suggested Devise a Stratagem, which is a fantastic mechanic (coincidentally, I believe the same dev that's leading the Magus made the Investigator, so I have high hopes).

I still can’t get over how monumental that change was to the class. Logan made a good call, but man it’s a big add after a Playtest.

A really awesome and amazing change, but considering how greatly it varies the Class combat wise it almost invalidates any of the Playtest data around the combat aspects to the point where the actual class basically wasn’t Playtested.

Maybe that’s a bullseye Logan can hit in his sleep over and over, wouldn’t be altogether shocking, but I think most people at Paizo would love to have a good first pass to minimize chance of missing on the real deal.

Although honestly I wonder if these classes are even intended to be the final at all or if they were just precursors that Paizo already playtested and made notable changes to already.

Then the Playtest is just to see if they missed anything glaring and to confirm that we all had the same issues they had (which makes honestly a lot of sense practically).

In essence, I’m sure our voices are being heard, but I also think Paizo probably already identified quite a bit of the issues internally and the amount of actual changes directly correlated to community only feedback might be less than we think and more that we corroborate changes they already planned to make.

[Temperans]

If its true they already detected those problems they could had just released the updated version for the playtest.

Testing an outdated model doesn't help to find valid data for a new model.

[Midnightoker]

Temperans wrote:

If its true they already detected those problems they could had just released the updated version for the playtest.

Testing an outdated model doesn't help to find valid data for a new model.

Identifying there is a problem and correcting it are two entirely different ball games.