The rules are a little fuzzy on this. By RAW, there is nothing that changes - it gives you 4 AC, you take a -2 to hit even with your bastard sword, and you can use it to give you Total Cover as a standard action. Don't forget that Max DEX limitation and the -10 Armor Check penalty.
All of that applies even to a large creature using a tower shield made for a medium creature.
But none of that makes any sense. How can a large ogre hide behind a shield that only gives Total Cover to ONE square? He can't. A clever GM would notice this and alter the rule to just giving Cover. Is that a house rule? Maybe - but if you look at the rules for Cover and Total Cover in the Combat chapter, you can clearly see that a 10'x10' creature CANNOT get Total Cover from a 5' "wall". So we have a rules conflict and a GM could rule either way, but the image of a large ogre trying to squeeze down behind this too-small tower shield should be the convincing factor - it shouldn't give Total Cover.
After that, we start to wonder why such a big guy takes that penalty to attack? If he had a Heavy Shield that is sized perfectly for him, it would really be the same size as that medium tower shield and weigh the same too, but wouldn't cause an attack penalty. Logically, it makes no sense. But the same logic would also have a problem with the fact that he only gets +2 AC with a large heavy shield and he gets +4 AC from a medium tower shield even though they're both the same size.
So logic doesn't apply here. Two shields the same size, one gives +4 AC with a -2 attack penalty, the other just gives +2 AC. There isn't a RAW answer for that.
Back to the actual question, can you TWF with it, there is no logical answer to this either. RAW says no because it's a tower shield. It is not listed on the weapons chart. If it were listed on the weapons chart, we could use the rules for larger or smaller weapons to figure out that it would be a one-handed weapon and figure out the damage. But it's not there so it is not a weapon so you cannot TWF with it. By RAW.
But again, logic argues with that. It's EXACTLY the same size as a large heavy shield and you CAN use TWF with a large heavy shield, so logically you can TWF with a medium tower shield for the same damage and same penalties to your TWF. In this case, it's not a light weapon so you are -6/-10 when you TWF with it. If you have the Two-Weapon Fighting feat, you're only -4/-4. Don't forget that it is still a tower shield so all your attacks take a -2 penalty, so you're really either -8/-12 or -6/-6 depending on whether you have the Two-Weapon Fighting feat.
As for me, GMing this situation, I would apply the logic to getting only Cover instead of Total Cover because the Combat chapter supports that. Then I would treat it like a tower shield for all other decisions, meaning you cannot attack with it at all. It's not logical, but it's RAW. If you want to attack with a shield, get yourself any other shield that isn't a tower shield. You won't have as much AC, but you will have much better attack rolls which seems like a good idea to me. If you REALLY must have that +4 AC from the tower shield, then you'll have to live with being unable to attack with it.
On a side note, is your bastard sword also medium-sized? If so, then you're taking an additional -2 to hit with that as well. If it's a large bastard sword, then you need the Exotic Weapon Proficiency feat to use it with one hand, but then it does more damage and there is no penalty to attack with it.
Mammoth.