Actually, I just found the answer and I feel pretty dumb. It's right there in the Animal Companion rules, the Link ability is what gives them the +4. Now, unless by some weird twisting of the rules only druids can take advantage of the special qualities animal companions get, and I don't think that's the case, anyone with an animal companion gets that +4. Oddly, the Handle Animal rules specify druids and rangers, when a Core Rulebook paladin can have a mount, which uses the animal companion progression chart and gets Link. That's why I was confused, my eyes glazed over Link but I noticed the line in the skill description.
I still don't know if a dip would give me access to all animals on the druid list, though.
The text remains unchanged from 3.5 D&D, where the paladin mount had a different progression from animal companions. 3.5 paladin mounts started with an Intelligence of 6, so Handle Animal wasn't needed as they weren't dumb animals (and indeed couldn't even be used anyway, as Handle Animal was limited to creatures of 1 or 2 Intelligence only).
The rules changed in Pathfinder, but not all of the text itself got changed.