I'm going to agree that this is a complicated situation.
The way I see it, if a player hasn't made any overt in-game effort to protect his character's spellbook, there's no reason to attack it. Unless you're the sort of DM who has bad guys constantly watching the PCs, and unless you're willing to have periodic theft of clerics' holy symbols and fighters' weapons, this is a class-specific screwjob in the works. Unless your rogue gets his thieves' tools pick-pocketed from time to time, or your sorcerer's staff of access to twice the number of good spells (ie. any) keeps getting stolen while he sleeps, or your bard's musical instruments turn up smashed every morning, your talking about a class-specific screwjob.
That said, if your players do make efforts, your job is to make those efforts worthwhile. Not necessarily by defeating them, but by challenging them.
If I have a party without Trapfinding, I have a game without traps (mostly). If I have a party without a divine caster, I won't throw a bunch of situations that need divine casting. If nobody picks up fly, I won't make many situations that demand it. On the other hand, if a player maxes out Acrobatics, I'll try to give him plenty of opportunities to tumble or balance.
Our job as DM is to enable our players' characters to do what the players want them to do. The magic is to make it not look like success is a given.