You're living in the era of Internet, aren't you?
Bugzilla, launchpad, technet, heck, even forum would do, provided you created a closed part, where people need to use their NDA personas.
As my esteemed predecessor said: you just look at the content flagged by QAcers, and read their comments. Hundreds of eyes make for better spot, er, perception check.
Regards,
Ruemere
How does any of that reduce the feeback cycle to zero time? You release it to your review people. Even with all the magic internet tools, you still have to give a window of time for them to review and give feedback. This can't happen until the book is finished with design. Layout can't start until the feedback cycle is completed. So you have to open a window for your crowdsourced reviewers to actually look at and return feedback. That has to be at least a month, but more likely two in order to gather enough feedback for the process to be useful. Then you need your developers to go over all the feedback which is at least a week, if not two. Then add another week for an internal review to make certain those changes haven't messed other things up. Now you give it to layout to finish the process.
All of that adds up to about 3 months added to the development schedule. Paizo runs on a tight schedule as it is. They simply can't afford to drag out the process that much longer.
|
Why not Cluster Source this.
I think you mean "Crowd Source".
There are plenty of people on the Rules forum reading and rereading Rules books (myself included). I would be more than happy to check stat blocks, especially if I get my name into a book. Give each possible monster to 10 people, 6 should respond, if 5 agree you should be good.
On another tangent though, I seem to recall one of the publishers saying that sometimes the errors aren't on their end, but on the printers.
Green Ronin maybe? I think mentioned they had already reviewed, edited, etc all their stuff and what came back in the actual final print (again after draft, preview copy, editing cycle 1,2,3, etc.) was something from an earlier release.
The most common of crowdsourcing solution in rpg world are patronage projects. I am extremely pleased with quality of Open Design products.
While I love Open Design projects, and I encourage anyone who wants to see how RPG books are made to join one, they are much smaller projects. It takes Open Design about the same time period to make a 110 page book as Paizo spends on a 256 page book. And even with all the patron feedback, errors slip through.
As it is, Paizo has experimented with public feedback. Every major rulesbook has had some portions go to public playtest. Even with all of that, some people are still unhappy with the gunslinger and ninja. Of course, most of those would have been unhappy just because Paizo made gunslingers and ninja.
So if you really want to help, provide feedback when the next playtest comes around.
How does any of that reduce the feeback cycle to zero time?The first step is to factor this into project management. I assure you that there is methodology for this. Plural. Methodologies.
The second step is to find out if you can split tasks into parallel jobs.
Paizo has to know a little something about project management in order to produce the volume of material that they do. And they do split things into parallel. Mostly, the layout team is working on book A while the design team is working on book B.
But as I've been trying to tell you, you can't put the review feedback in until after final design is turned over. You can't start layout until feedback and changes have been finalized. (You can, but you'd spend almost as much time re-laying it out after the changes are made.)
So you can divide your manpower to work on other books while you wait for "patron" feedback, you still have to start the book 3 months earlier than before. Which makes it even longer before you get a return on investment, which Paizo simply cannot afford at this time.
Let's agree to disagree then. My knowledge of Paizo structure, budget and plans is limited and so I cannot argue with you.
Peace. And out.
Hey, no problem. I speak as a software engineer who put together an RPG PDF with another software engineer. We tried implementing all sorts of version control things into our process. We were frustrated to no end when we realized there was no way to import the XML stats our monster builder program created into an InDesign template. We also frustrated our layout person by continuing to try and tweak things after we had turned it over.
So I can't say I'm an expert on publishing, or that I have inside knowledge of Paizo's business processes. I can say I have enough experience to know that you can't treat books like software development. I also trust that Lisa and others at Paizo who have 20+ years of experience in the RPG publishing industry are skilled at things like project management. The fact that they manage to get a dozen 96 page adventure books out per year is remarkable. And they do supplements, modules, and rules on top of that.
So I admit I get passionate because I've at least glimpsed both sides and have great respect for the amazing job Paizo does.
Quality Assurance is not about fixing stuff. That's a designer's job. QA person aka QAcer is meant to submit spotted problems.Someone offered a perfect solution to this...why won't Paizo go through with it...?.Time constraints? Quality control? Probably others.
I personally don't think the community are as good at game design/development as we think we are.
I understand that - my post was unhelpfully brief and inaccurate, I guess. What I meant is that we're not as good at spotting errors as we think we are. As Sean pointed out later - half the flagged problems are not going to be actual problems, they're going to be misunderstandings of the rules on the part of the QAcer.
My comment about quality control was referring to quality control of the volunteer proofreaders. I regularly make clueless observations outside of my field and this is no exception - nonetheless, I suspect that the extra time and effort the Paizo staff would need to expend putting something like this in place and ensuring it continued to run smoothly and on schedule would be far more efficiently spent doing the actual QA themselves.
If Paizo were large, with the same number of expert, potential volunteers that Microsoft have access to - well then it might be feasible. I do not have the same confidence that you do that the Paizo community are all that good at interpreting the rules. We're keen, sure - but enthusiasm is not necessarily a plus here.
Crowdsourcing, i.e. using volunteers to do simple stuff en masse is a time honored practice of producing consumer software. And it works for Microsoft, Canonical, Mozilla Foundation, Opera etc.
I dont think examining RPG rulebooks for potential inconsistencies, ommissions, errors or other issues is simple. The other big problem with volunteers is that there's nothing to stop them quitting at the drop of a hat. Volunteers work best when their tasks are both simple and non-essential (unless you have a very large pool of potential, appropriately skilled volunteers).
The way I see it, Paizo designers lack time to read stuff thoroughly... If I, a mere aficionado, can find fishy combos at a single cursory reading, that tells me that someone else didn't have time to that reading. It is nothing unusual in a fast paced work environment. And the solution of mine is just as typical.
Obviously I'm not having a go at you - I'm very confident you're better at doing such things than I am. Nonetheless, I dont accept that something you find fishy is necessarily something a designer should spend time reviewing. In my experience, the things put forth as 'glaring problems' usually have their fair share of defenders - suggesting to me that there is an enormous subjective element in the things the community would be interested in reviewing.
If this was solely about looking for spelling errors or homonyms I think a pool of volunteers could well be useful. I dont think that's what people are crying out to have fixed though.
One thing that I feel I have to say is that by and large I really enjoyed both Ultimate Magic and Ultimate Combat, the latter moreso. One of the problems that comes up is that both of them had...well let's just say certain handicaps out of the gate.
UM was the first focused supplement and they introduced some interesting new ideas but also some that were...well not so good (many of the monk vows and/or a lot of the witch hexes). The Magus was frankly awesome and is probably one of my favorite classes and I also like the ideas for the new magic system. Most of the complaints that I saw for UM were either screams of power creep or some complaints of things that were pretty silly (as mentioned before, many of the witch hexes or the oaths for the monk).
UC had a different problem, it had a few things that people complain about, oriental material and guns. There are people who would complain about those being added regardless of how well or poorly the material is integrated. While I might not use everything listed I like the options presented. For example while I might never use some of the alchemical vehicles listed I like that the rules are there for them if I want to try that kind of campaign or if I want to play with that sort of thing conceptually.
And how do you determine who is qualified to give this sort of playtesting evaluation?Presumably, the same way you determine who is qualified to write the book in the first place.So, by starting them on small writing projects, then graduating them to larger ones once they've proven they have the necessary skills to manage a large project on time and do it well? Clearly I have time to train an entire team of playtesters in addition to everything else I do....
I'm thinking that an insurance actuary would be a better bet than some morose dude who thinks he's qualified because he goes around winning the internet all the time. The problem with that is they'd probably need a whole lot of money.
Kobold Press
|
| 3 people marked this as a favorite. |
The most common of crowdsourcing solution in rpg world are patronage projects. I am extremely pleased with quality of Open Design products.While I love Open Design projects, and I encourage anyone who wants to see how RPG books are made to join one, they are much smaller projects. It takes Open Design about the same time period to make a 110 page book as Paizo spends on a 256 page book. And even with all the patron feedback, errors slip through.
This is exactly true.
Open Design projects are heavily playtested and proofread by a smallish group of motivated people. That means it takes months longer to organize, play, write up, evaluate, and fix an Open Design book, and because of that work Open Design produces a high-quality book.
But the size of the audience is limited to a relative handful of patrons, not thousands of Paizo fans and playtesters. As SKR points out, even bad reports require time to read and evaluate.
At a certain point, the extra time and effort required to manage a project like this isn't viable for a company with a payroll, rent, overhead, etc. Open Design can do it because it is small and not profit-driven. I think it's probably unreasonable to suggest that Paizo become even more transparent than it already is.
Honestly, I stand in awe of the Paizo Beta and playtest documents, and the results they produce. Nothing's perfect, people, but damn, Paizo does really, really good work.
| 1 person marked this as a favorite. |
And how do you determine who is qualified to give this sort of playtesting evaluation?
Phrenology