Mulgar, could you walk me through a situation where "don't play with him" isn't an option. Because the way I understand things, it is always an option.
Its not always a particularly viable one.
Tiny Venue: There's one table. He's at it. Not playing with him means not playing
Slightly larger venue: There's two tables. You want to play high, you may have already played the other scenario. A table not happening means they collapse into 1 and you have to go home.
Even at a larger venue the celestial bodies of scenario you can play, character you want / need to play to make that eyes of 10 game next week can bring you into the same orbit and leaving when they show up can cause a scene.