In mechanical terms 4 levels of Paladin with Oath of Vengeance bolted onto a high Charisma Bard works really well. The fact that this particular PC had the Dirge Bard archetype, which grants fear based powers and necromantic spells, made the combination even stranger from a roleplaying perspective. The fact it was unusual and unexpected is part of what made it fun though. A man who is immune to fear but able to frighten even ghosts with his righteous fury was fun to play.
It seems pretty natural to me that a deity (or perhaps just some universal force of Good) could suddenly fill a PC with holy power. At least in this PC’s case the weapons and armor he used didn’t change any. He just devoted himself to “Stag God”, began declaring oaths of holy vengeance, and start smiting evil a lot. Folks might not expect a guitar playing Paladin who drinks, curses, and has a monkey familiar, but none of those things are actually against the Paladin’s Code of Conduct.
By the end of the campaign the PC had 4 levels of Paladin and 13 levels of Bard, but I think folks tend to remember him as a Paladin.