In short, I love the idea.
My group has also been playing APs over the past couple of years, and I find that the preponderance of full casters can often be frustrating. Maybe boring is a better word, actually. The availability of divination and teleportation spells can seriously curtail the storytelling options available to a DM. (That or the DM has to escalate the magic of the setting overall to pre-emptively thwart the PCs' use of magic.)
This statement you made perfectly sums up the notion:
"It occurs to me that without the reality-altering crutch of excess magic, the PC's might be more inclined [to] developing and using their skills and [to] coming up with more creative ways to deal with challenges than hand-waving and finger-wiggling. The goal is still fun, after all."
I find the game to be very well balanced in the low levels, regardless of the PC's classes. As a DM, remember the following rules: (1) the goal is for as many people at the table as possible to have fun, and (2) the DM is always right, as long as "being right" doesn't interfere with rule 1. In that vein, set up encounters using your best judgment... and then fudge the numbers "behind the screen" if an encounter turned out more deadly than you intended.
Off the top of my head, the only Pathfinder "trope" that you may run into trouble on is the availability of healing relative to the expected number of encounters in a day of adventuring. Without clerical channeling, or easily-available wands/scrolls of healing (per your statement about magic items being rare), you will likely find that the party can handle fewer encounters per day than the relevant Core Rulebook section says a party should be able to.
