I've been doing some work on some home-brew characters recently, and have got to the point where I'm about ready to do a proper play-test.
However, this has got me wondering on the best way to play-test a Homebrew character.
I'm guessing that official characters are played through the entire AP (as they'll also be testing the AP itself), by lots of people etc.
For a Homebrew character, the ultimate aim is just to play them through the AP once- it would feel odd to have a playtest that was longer than the actual shelf-life of the character.
What would people recommend? - is it worth picking a scenario or two here and there, and artificially generating a typical version of how the character would look at that point? what are the things that you'd be particularly on the lookout for
I created 4 characters for RotR, and launched into the AP without play-testing. One was spectacularly under-powered, and we had to errata him, another was probably slightly over-powered, but once we'd started nobody wanted to take that hit. One was basically ok apart from a single broken power. The fourth was fine in theory but always rolled terrible dice - overall the experience was ok, but it was obvious that these weren't of the same standard as the "real" characters.
I'm not sure how much play-testing would have been needed to spot these issues, so i thought I'd see what others thoughts were.
When I made my six characters for RotR, my girlfriend and I went in to the experience expecting that the actual play through would be a playtest, and I made sure that we had no illusions about that before starting. Despite us not wanting to take the hit several times, she trusted me and I made several power-level nerfs to multiple characters, both hers and mine.
I think the most important note is, if you are using a game both as a playtest and a fun experience, make sure everyone involved knows that, and that they trust you to have the final say on power level changes.
Also, be sure to talk to your group very frequently, possibly even having a regular update schedule. One thing we did was to have a discussion session after each full adventure, where we proposed and enacted changes. By having that regular schedule, we were able to avoid surprise nerfs, and made it feel much more natural (it also helps that we play WoW regularly, and are thus used to having powers nerfed seemingly at random). There were occasions when we had to make an emergency change in the middle of the adventure (I think the big one was when I determined the power of one character should not be able to be changed from bury to discard on the base card), but that was always made with mutual agreement.
If you don't think your group will want to play in that way, here's the best recommendation I can give:
At minimum, run the base set adventure. That will reveal the obvious stuff, and is a wide enough sample that you can avoid a decent bit of die randomness. After that try running at different points. The keys I'd try to hit are Adventure 2 (After a few power feats), Adventure 4 (Right after role card), and Adventure 6 (Fully powered). I choose those, as they are probably the most easily identified inflection points in character power.
Also, when you do your testing, I recommend testing them with existing characters, not just your homebrews. The reason for this is to create a control sample, and make it easier to judge relative strength as opposed to absolute strength. If you really want to create a good character across the spectrum, you'll want to do playtests of different sizes as well, since characters can play VERY differently depending on scale. However, if you're only planning one playthrough like you said, you can probably just focus on party size matching the one you'll be using.
Obviously nothing here is official, but that's my lessons learned from prior experience.