Posts
As a game designer of some of these scenarios, this topic is of great interest to me.
I think a number of you folks have nailed the reasoning behind the variance.
1. scaling the tiers can really wonk a scenario.
2. diversity of players and playing styles can really wonk a scenario.
3. increasing or decreasing the mount of PCs at a table can really wonk a scenario
4. Having a table with mixed levels can really wonk a scenario
5. different GMs can really wonk a scenario.
Neil Shackleton's suggestion to have a solid guide for adjusting a scenario for the number of players is spot on.
I also think Org Play tends to encourage character optimization and therefore, writers should err on the side of optimized encounters. That said, there might need to be clearer guidelines for scenario writers as to what an optimized encounter is.
I have another idea as well.
Create a solid guideline for GMs to tweak a scenario. And make those guidelines very very simple. The objective would not be to solve the problem (which is never going to happen nor should it, there should be diversity between different scenarios, otherwise everything becomes the same and therefore, boring), but instead should take a little of the sting away when the CR of an entire scenario seems way off.
Something like a once per game modification, which can only be applied under certain conditions.
For example:
If more than two characters die, the remaining characters gain a +2 "avenge the fallen" bonus for the rest of the scenario.
or
If the PCs complete two encounters with minimal damage (all party members have at least 3/4 of their total hp left.) Then play the next encounter up a tier. If the PCs at at the top tier, give their opponents a +2 bonus to every d20 roll.
Note: the aforementioned solutions likely need a little more thought than I gave them, they're just examples I used to showcase the concept.