Monsters intelligent enough to spot a caster are intelligent enough to run away if they think that "the horrors of hell" are about to be unleashed on them. The argument is that their fear of things other than death is so great that they will risk death to... fight the being who can inflict the horrors? They would just run away if they were that afraid.
Monsters are, in general, not incredibly experienced at fighting things with the lethality of an average PC party. Also, monsters and villains, like almost all evil beings, are driven by personal gain and self-sacrifice for the betterment of "the team" isn't at all logical. In the case of animals defending their homes and young, the closest threat to the home is the focal point.
On the uncommon occasion you run into a group of intelligent beings who are willing to sacrifice themselves for victory and recognize the caster as being undenaibly more deadly than the barbarian with a two-handed sword in front of them, and not being too scared to fight that caster even while thinking it is so more deadly that the guy about to cleave them in two... sure, go for the caster first.
Edit: Came across as a bit dogmatic. I mean in general. Obviously doing in sometimes would be fine, but there's no reason to think it is the logical conclusion for a majority of enemies.