Per SKR here:
If you could potentially be making an attack with it (or an AOO with it), you are wielding it.
So to address your questions:
Dueling Mastery says " As long as you wield only a single dueling sword in one hand, you gain a +2 shield bonus to your AC"
If you are just holding an item(or weapon) in your off hand do you count as wielding it. For instance an archery Ranger holds his bow to draw a dueling sword for melee. Does he count as wielding a single dueling sword?
In the example with your Ranger, he would still keep the AC bonus. He is holding the bow, because he cannot wield it (it requires two hands to wield - i.e., to effectively make an attack with it), but he is wielding the sword.
If he had a dagger in his hand and drew a sword to wield it, he would have to make a choice: He could either continue to wield the dagger (in which case he would threaten with it), or he could simply hold the dagger (in which case he'd get the AC bonus, but could make no attacks with the dagger, including AoOs).
Two Weapon Defense says "When wielding a double weapon or two weapons (not including natural weapons or unarmed strikes), you gain a +1 shield bonus to your AC." If you merely hold a weapon and single attack, or full attack with one hand, or full attack with both weapons with just BAB do you benefit from TWD?
Take a look at the discussion of the defensive property a few lines down in the thread I quoted. That property specifically says that you only get its benefit if you use the weapon. Compare that to TWD, which says that you only need wield the weapons.
So it doesn't matter whether you attack with one weapon, both weapons, or even neither weapon - as long as you declare that you're wielding both of them (and actually can wield both of them), you get the TWD bonus.
The potential penalty comes in if something comes up where you need a free hand; if you've stated that you're wielding both weapons then your hands are both occupied.