Depends on one thing, Would everyone agree that it is well within the rights of the GM to decide that the Wizard has to memorize spells that he says or the the cleric has to use channel energy when he says? Because they are all class features of the character.
It seems to me taking control of a familiar is just like telling the cleric that he just used his channel energy to heal everyone. even if he had no intention of doing so himself.
