It is again, the same as my 8 int fighter example. Sure, you can pretend to be an idiot savant, or argue that you'd have some insight into this puzzle because of some misc thing in your backstory. But the end result is usually people just decided they wanted more physical stats and paying attention to the negatives is inconvenient.
You say paying attention to the negatives is inconvenient. I say that they are paying attention to the negatives and you just think the negatives aren't enough. There's a penalty associated with having a low charisma: you're worse at bluff, diplomacy, intimidate, perform and UMD without some special feat or trait to change things. Saying "I'm going to not engage in social situations because I'm bad at talking to people" is paying attention to those negatives and makes sense both from an in character and gamist perspective.
Now you can say those penalties aren't enough and I'd agree with you, but phrasing it as someone ignoring the negatives is just disingenous.
The point does remember me a situation I'm tired of seeing. The "party face", where one guy have high cha and social stats and the other party members don't even bother to talk because "let the guy with the high diplomacy do the talk, every talk, like ever". Not really a fan of social skills for that.