There are LOTS of stuff you can do about this. One of my favorite is to have a fake-BBG - an illusion works the best or a polymorphed lackey. The BBG could then step in BEHIND the party while THEY are in the silence and hurl spells aplenty.
How about an anti-magic shell, no 3d or lower spells work in it.
How about the BBG has a chime of dispell silence with limited charges available, hit it inside a silent radius and the silence goes away.
HAve your BBG take the Silent spell Feat and memorize Dispell Magic.
Give your rooms very high ceilings and have your BBG up on a levitating or flying above the silent radius.
The possibilities are endless.
Just my 2p worth,
Brian