When I played an Outwit ranger, I ran with a bastard sword and high Str. The only reason it wasn't a greatsword was because I valued the ability to use combat maneuvers sometimes over the ability to do piercing damage. Overall, I think it worked out fine, except for the fact that my horrible dice luck invariably caused me to roll low on everything.
Effectively always having a buckler raised was a nice way to give a bit more defense while still having that d12 damage.
Outwit as a way to maintain some AC while using a 2H weapon? You might be on to something here. "Witty" isn't the first word people think of with a greatsword but it could work. You'd be oddly defensive about who you declare prey though, since it powers your defense while for the other styles it powers the offense.
As for sightings in the wild, I haven't seen anyone stick with it for more than a few sessions either. But I think that's because they tended to go into it with the idea that they'd be the RK person. But that's not like a full-time party role, it's something you do on the side or divide across the party based on who has Wis and Int. So it feels kinda sour compared to more tangible combat benefits from Flurry/Precision.
But a greatsword with AC and Feint bonus, that sounds playable. Maybe not top-tier, but good enough to function and not something you see all the time.