Just wondering people's opinions.
Personally, I think the mount would be a lot better than the spirit except that the mount is limited on where it can follow you. In general it cannot go into dungeons, buildings, etc.
personally i would go with the celestial spirit. Anything that cant go with me into most buildings/dungeons is not a useful class ability in dungeons and dragons. In 3.5 my group always replaced the special mount with something, which varied from game to game. Its sad because I like the idea, but it just isnt practical unless the game focuses on being out in the open.
It's probably also fair to ask your DM what they think.
I started running Curse of the Crimson Throne (an urban campaign), and one of the players created a Paladin geared towards mounted combat. I recommended he take Celestial Spirit instead, a decision he has not regretted.
Another anecdote: I made a mounted paladin once who used ride by attacks all the time, but I got totally chewed out by the other players because they didn't like the fact that my paladin wasn't protecting them.
I'd say the mount is potentially more powerful, but also more situational. It's a tradeoff.