So my main question is: If you are choosing to deal non-lethal damage with a weapon (with or without penalty) and you are using smite (good/evil/etc) is the extra damage from smite still lethal damage?
I know of the Redeemer and it's merciful smite, but that also removes the fairly serious -4 attack penalty as well as everything else smite does.
As a follow up, apart from the merciful enchantment is there anyway to make bonus damage from class abilities & feats non-lethal?
The additional damage from the Smite is of the same type as the damage of the attack triggering the smite.
If you are using a longsword lethally, then the smite's additional damage is lethal slashing damage (added to the original damage roll). If you are using a mace nonlethally, then the smite's additional damage is nonlethal bludgeoning damage. If you are casting Ray of Frost from a one-level dip in Sorcerer, then the smite's additional damage is lethal cold damage.
In all these cases, you add the bonus damage from Smite to the original damage result. For example, while Smite does allow you to ignore DR, it does not let you ignore resistance, so in this case adding the extra smite damage to your Ray of Frost may influence whether you pierce cold resistance.
All additional damage on attacks functions this way. Be it Smite, Sneak Attack, Inspire Courage, or Power Attack. Anything that increases how much damage you do without specifying for itself whether it's a specific kind of damage uses the same kind of damage as the originating attack.
Thanks for the clear and consise answer. So in the case of options that do specify a damage type (such as elemental fist) does the extra damage do lethal, unless of course the description for that ability states otherwise?
If it's a melee attack, I'm pretty sure you can always opt for the damage to be nonlethal, but with a penalty. It's ranged attacks and spells which don't give you this option.