Testing methodology proposal

[ruemere]

i. Basing on the structure of playtesting feedback, here is a suggestion on how to conduct open playtests in the future.

1. Provide base class material (already implemented).

2. Provide builds created by class designers. Builds should include items and several versions (for different levels). Builds should not be optimized.

3. Discuss builds and alter them as per player feedback. A few builds intended for abusing class mechanic should be added (community feedback would be most helpful in this regard).

4. Provide challenges (a list of sample encounters or Bestiary monsters). A sample encounter should be sketchy - a list of opponents, starting distances, general description of terrain.

5. Evaluate playtests.

ii. The reasons for this proposal

1. Two weeks are unlikely to provide ample time for high level tests. And, as per our experiences, stuff tends to break down more at higher levels.
Sample builds should facilitate casual playtest in campaings.

2. Quite a lot of feedback provided was rendered somewhat useless due to changes introduced by designers. The on-the-fly changes are necessary, they are also appreciated by the community, however they also mandate reevaluation of tests. Presence of sample builds and challenges is likely to ease the pain of changes.

3. While comparing three forums for each of three rounds, it is pretty evident that quite a lot of enthusiasm was lost over time. It's to be expected as playtests require quite a lot of preparation on both player (new rules) and GM (rules and encounters). By adding sample content community users would be able to devote more time to playtest.

--
Hopefully, this short commentary will be of some use.

Regards,
Ruemere

[jreyst]

I agree. I think this "playtesting" phase is really not anything of the sort and more like a "see if you guys notice anything we obviously missed", which is still certainly useful for catching glaring flaws, but it is not really any sort of structured playtest. I'd have liked a lot more structure to this but in all honesty, if they took the time to plan out the structure as much as I'd have liked we'd probably still be in beta for the core rules and no one would have a book yet. Being that they are a business, business concerns need to be realistically considered. Sometimes "good enough" is good enough and "perfect" is economically unfeasible (sadly).

[caith]

I think a core portion of this playtest was exactly to save the designers time, by having the players do that work. That said, this is a very useful proposal. With this work done, the playtest would have been much more effective, with a larger sample of data available.