I really don't get a chance to play characters as I am generally (over 90% of the time) the game master for my group(s). I have seen it in play and while it certainly has some negatives that you pointed out. It gains in that you can actually prepare all combat spells you will need and if you truly need a utility spell to solve a problem you more than likely can nab something off the cleric druid or bard list to solve said problem.
There is also the trick where you use the preferred spell feat to be able to spontaneously cast any one spell you want off the druid cleric or bard list you truly want access to all the time.
I look at the archtype as the ultimate generalist wizard, and you aren't entirely screwed if you lose access to your spellbook. Furthermore if you want a familiar there isn't really anything stopping you from dipping say sorcerer and snagging a bloodline familiar or doing the whole animal ally thing and getting an animal companion instead. In addition to all this if you walk on the darker side of the alignment spectrum you have access to desecrate so your undead will be stronger than the average necromancer and you have the distinction of being able to start making undead as early as level 3 (using lesser animate dead off the cleric list) meaning with enough downtime and a small monetary investment (or none if you use false focus as a feat) you can have a compliment of 12 skeletons or zombies to fight for you which pretty much wins any scenario you might face at those levels.
The once per day +4 CL boost is excellent for augmenting summons spells which are a little weak on duration early on. Or for turning that burning hands spell into a save or die at level 1. Or making mage armor last for 33%-50% of your adventuring day. Finally as early as possible I would craft or find someone to craft a Staff of Entwined Serpents which gives you something to replace cantrip spam early on.