| 1 person marked this as a favorite. |
Let me begin by reiterating, needlessly, the first rule of the boards, which is don't be a jerk. Let's keep this thread on topic, without resorting to bitter jabs or oneupmanship.
By its nature, we'll have to look at things that people felt negatively about. Let's all approach this from a position that we want to avoid those situations in the future, rather than extort some apology or acknowledgement about the past.
Now, I have taken part in various playtests since before Pathfinder was just an idea, back in the Alpha days. I think they are one of the best things about Paizo, and really showcase how different they approach their business. As such, I like to see them be as successful as possible.
I'd like to start out by listing five points I've observed.
1. More frequently playtested material is better than less.
2. More thoroughly playtested material is better than less.
Maximizing these two facets within reason is obviously a desirable goal. We all understand that you can't give ALL of the material into the playtest. I want Paizo to realize that the material that is given, and worked on, typically seems to come out the best and most usable in my games. If you compare the alpha and beta playtests in Core to the mythic adventures playtest, you see that a lot more material was playtested, and combinations of such were better vetted, often going through multiple updates. Same goes for the ultimate and advanced releases. The materials that returned in a final 'feedback altered' form generally were more successful. Success in this case being a subjective term for 'useable with less to no changes in my game'
3. Acknowledgement of feedback is important. For instance, in the mythic playtest, I suggested surge be a free action. In the final form, it was still a swift/immediate action. That's a fine decision, but at no time did I feel that Paizo had considered, decided against, or even noticed my suggestion. I also got the same impression reading most of the other feedback. It didn't feel like we were being listened to, regardless of the end decisions. More importantly, that was NOT the feeling I had in previous playtests.
4. Timely communication is important. The mythic playtest was updated exactly once that I'm aware of. Then, six weeks after the fact, we were informed that the playtest had concluded. I'm not going to lie, that felt borked. I felt like I'd been wasting my time, when even a simple 'hey, playtest closed' note would have let me know to stop. I don't expect pay or accolades from playtesting, but I hate feeling like I was so insignificant that even the barest of common courtesies was too much hassle to extend. And when it was brought up, those of us who brought it up were treated as though we were entitled pricks. That's not right or fair. A couple days late is understandable, but a month and a half isn't.
5. Explanations are appreciated. We're all nerds or geeks here. When designers have had and taken the time to explain how they envision things working, or how they were used in their own games, that really gives valuable insight into the intent of abilities or features. I get geeked out when I can read that stuff. It builds excitement and a sense of community for me. I know that time constraints keep it from being possible always, but whenever you guys have the chance, that stuff's gold.
Well, that's it for right now, at least from me. What sort of suggestions for future playtests (on both Paizo and playtester side of the aisle) would you guys like to see?
| 3 people marked this as a favorite. |
I only watch these things from the outside, but I think it's worth bearing in mind that when you're playtesting something you're not co-designing it.
This isnt a direct response to your post as such, but merely to some of the complaints I've seen after-the-fact. People have seemed quite upset about the fact that their suggestions as to how things should be improved were either ignored or dismissed. It seems to me that although such suggestions are probably welcome, they're not the actual point of a playtest.
My (quite possibly mistaken) view would be Paizo are saying "Here's where we are at the moment or here's somewhere we're thinking of going. Try it out and let us know if there's any problem." rather than "Tell us how you think we should change it".
Timely communication is important. The mythic playtest was updated exactly once that I'm aware of. Then, six weeks after the fact, we were informed that the playtest had concluded. I'm not going to lie, that felt borked. I felt like I'd been wasting my time, when even a simple 'hey, playtest closed' note would have let me know to stop. I don't expect pay or accolades from playtesting, but I hate feeling like I was so insignificant that even the barest of common courtesies was too much hassle to extend. And when it was brought up, those of us who brought it up were treated as though we were entitled pricks. That's not right or fair. A couple days late is understandable, but a month and a half isn't.
I didnt see the issue play out, but this certainly seems reasonable to me.
Let me begin by reiterating, needlessly, the first rule of the boards, which is don't be a jerk. Let's keep this thread on topic, without resorting to bitter jabs or oneupmanship.
4. Timely communication is important. The mythic playtest was updated exactly once that I'm aware of. Then, six weeks after the fact, we were informed that the playtest had concluded. I'm not going to lie, that felt borked. I felt like I'd been wasting my time, when even a simple 'hey, playtest closed' note would have let me know to stop. I don't expect pay or accolades from playtesting, but I hate feeling like I was so insignificant that even the barest of common courtesies was too much hassle to extend. And when it was brought up, those of us who brought it up were treated as though we were entitled pricks. That's not right or fair. A couple days late is understandable, but a month and a half isn't.
As explained elsewhere, this wasn't Paizo dissing people. This was a confluence of setbacks, including medical issues, affecting the person responsible for managing the playtest.
Taking a page from software development, having a single person as the only one who can do something is a problem waiting to happen. So, perhaps lesson learned in this case would be that a backup person charged with at least managing communications should be assigned to the project. Paizo is a small company, and can't put two people on some of these projects full-time; that's understood. But someone to manage damage control when there's an issue is probably a good idea.
As explained elsewhere, this wasn't Paizo dissing people. This was a confluence of setbacks, including medical issues, affecting the person responsible for managing the playtest.
As explained elsewhere, this excuse falls more than a little flat. I find the argument "we didn't have time" incompatible with dozens of Facebook postings (to say nothing of other postings) including long accounts of gaming sessions played during these six weeks and pictures uploaded from those session. This wasn't a day or two, or week or two in which something was buried amid emergencies. It was a month and a half, and even then the actual post that announced the closing of the playtest had to be prompted by someone else posting on the subject and bringing it to the forefront.
As I said originally, I've never held that the Paizo staff owes me anything personally. I also don't believe that this was some malicious plot to waste everyone's time or disrespect playtesters. I do think however that this goes a little beyond offering a "we had no time" when a post announcing the end of the play test would have quite literally taken less than 5 minutes to type up. Even if JB was the only designer possible of typing up such a post (which seems unlikely) the suggestion that it was an impossible or even enormous hardship to put out that much effort over the course of more than a month rings a little bit hollow. We are talking about 5 minutes out of more than 60,000.
It strikes me as more likely that officially ending the playtest was simply not a priority. It was something that fell by the wayside and was forgotten. When you do have illness, deadlines, and vacations posting on a message board isn't your top priority. That's understandable, but it should be made clear that it was more than a little bit aggravating to a large number of people who each devoted hours of their time each week playtesting these rules and dutifully typing up posts reporting their experiences and findings. No one expected anything in return for doing so except to have their feedback acknowledged in some way, and instead what they found out is that it was a complete waste of their time. Feedback on a playtest that was never used because the playtest was long over. Hundreds or thousands of collective man hours down the drain because saving them that time and effort wasn't in any way a priority.
Honestly, it was something I hadn't seen out of Paizo before, and it's the sort of thing that leaves a bad taste in everyone's mouth, especially when instead of a response that says "my bad" which everyone can move on from, we collectively got a boat load of snark and an excuse about not having 5 minutes to spare. If we are talking about improving playtests - the subject of this thread - that shouldn't be repeated. It's off-putting to fans who were loyal enough and excited enough by the prospect of Paizo's latest project to devote a ton of time and energy towards it for no good reason. It's the sort of thing that creates bad press out of something that should be nothing but good press for Paizo - namely playtests.
It's also worth noting, in the course of discussing communication issues with regard to the Mythic Playtest, that it's ending wasn't the only problem. There were posts and announcements about how the playtest was to start in September - but the month came and went without any update. Instead it pushed forward into the second week of November before anyone heard word about it.
I can think of tons of reasons for a playtest's release to be pushed back that are all a thousand times more valid than a failure to close it down on time. I'd also observe that if a playtest starts late no one has really been harmed in any way. This is sort of a molehill next to the mountain that was the failure to end the playtest. That said, I remember finding it somewhat shocking at the time that a communication breakdown of that size occurred in a company that has otherwise been extremely good at communicating with its fans - especially since there were other blog posts at the time in which a comment about the push back could have been thrown with ease.
TL:DR - Communication about the start and end of the playtest was pretty bad regardless of the reasons behind it, and moving forward I don't think anyone can say that improving communication is a bad thing for any of the involved parties.
I only watch these things from the outside, but I think it's worth bearing in mind that when you're playtesting something you're not co-designing it...
Try it out and let us know if there's any problem." rather than "Tell us how you think we should change it".
I agree that many playtesters need to check a bit/lot of their ego at the door. They aren't developers. On the other hand, the point of the playtest is to solicit multiple opinions. I think that's where the communication comes in. I don't mind if you do things differently, as long as I know you heard me.
Confounding Blades (Ult Combat) vs Slow Reactions (Core) and Mythic Bond (Mythic Adv) vs Hunter's Bond (Core) are things that wouldn't have happened if the abilities had been playtested. Someone would have caught the overlap.
Tinalles makes a good point. Clear(er) guidance on what is being looked for would definitely help, it gets back to that communication thing.
Sean K Reynolds has been known to write up some of the best explanations for entries into the RPG Superstar contest. While I've never entered, I certainly feel that if I were to, I'd read over those pieces to make sure that my concepts were fitting within those rules. Something similarly specific aimed at clarifying what we should be providing in a playtest would definitely be a useful direction to head in.
I only watch these things from the outside, but I think it's worth bearing in mind that when you're playtesting something you're not co-designing it...
Try it out and let us know if there's any problem." rather than "Tell us how you think we should change it".I agree that many playtesters need to check a bit/lot of their ego at the door. They aren't developers. On the other hand, the point of the playtest is to solicit multiple opinions. I think that's where the communication comes in. I don't mind if you do things differently, as long as I know you heard me.
Confounding Blades (Ult Combat) vs Slow Reactions (Core) and Mythic Bond (Mythic Adv) vs Hunter's Bond (Core) are things that wouldn't have happened if the abilities had been playtested. Someone would have caught the overlap.
Tinalles makes a good point. Clear(er) guidance on what is being looked for would definitely help, it gets back to that communication thing.
Sean K Reynolds has been known to write up some of the best explanations for entries into the RPG Superstar contest. While I've never entered, I certainly feel that if I were to, I'd read over those pieces to make sure that my concepts were fitting within those rules. Something similarly specific aimed at clarifying what we should be providing in a playtest would definitely be a useful direction to head in.
I can't imagine more communication doing any damage. As I said, I dont really participate in the playtests (I dont know the rules anywhere near enough). It surprises me that there's a perceived communication issue, given how open and quick to respond Paizo generally are. Worth being aware of next time, I'd guess.
I'm sure they could have. The trouble with stuff like that is it probably isnt a major part of anyone's job description to 'keep an eye on whether we've told people the playtest has finished'. That's the kind of thing (particularly if there were health issues, delays or other unexpected events) that can easily slip through the cracks.
I'd bet good money Paizo would wish someone had made that kind of post. However, I dont think it's some terrible slap in the face. It's just one of those things which happens from time to time. (I'd much rather this kind of thing gets missed than a file doesnt get shipped to the printer within a deadline or any number of other trivial little tasks which need to be done day by day).
| 1 person marked this as a favorite. |
As with each situation I think there are lessons to be learned, and room for improvement. But let's not think that "playtest" happens in a vacuum. Part of the feeling I get from those protesting the loudest is that the definition of "playtest" is something like "drop everything else," and that is simply unrealistic.
Perhaps the key is to set better expectations at the frontend, or perhaps not even have the playtest, if the definition of "playtest" is supposed to be what I mentioned above.
So, yea, always room for improvement, but let's not go overboard here.
You might not find it worthwhile to try and improve it, Elorebaen, but there are plenty of threads that don't deal with improving the playtest. This one is about how we can improve it. If you don't have anything to add to that, feel free to not.
So far the big suggestions are all variants on improve communication. Improve specificity in what Paizo is looking for, improve communication during the playtest, make sure the close of the playtest is communicated, etc.
On that very last one, I wonder, does Paizo know before the playtest starts when they intend to end it? Providing that timespan up front if possible seems like it would have headed off the entire issue from Mythic.
Well, it's worth pointing out that there was a posted end-date for the Mythic playtest. The problem was that it was posted on a blog entry from months before and on the original product page where you downloaded the rules. Neither were pages people were likely to look at more than once, and there was no further heads up about those end-dates.
Publisher, Chief Creative Officer
|
| 6 people marked this as a favorite. |
Thanks for the honest feedback. It's pretty clear we could have been more communicative regarding the wind-up of the Mythic Adventures playtest, and I apologize on behalf of the company if you feel that your efforts were wasted. We will endeavor to improve communication on the upcoming Advanced Class Guide playtest. This thread has provided a lot of good food for thought on that score.
Thanks for sticking around and using a frustrating experience as the basis for providing constructive criticism.