There's a big player imbalance with how much fatigue nerfs a Barbarian when it takes away Rage. At level 1 or 2 it doesn't do too much relative to other classes, but by level 5 or 6 you're losing 1/2 of your class abilities, by level 10 probably 3/4 of your class options are gone. Is there any other class that is wiped out/punished so harshly by a single common condition?
It would be like telling the fighter to give up 3/4 of their feats when enfeebled, or a wizard loses half their spell slots when stupefied.
The other side of the problem is there's almost no way to get rid of fatigue outside of rest. There are consumables to aid with most conditions, but nothing that let's you, for example, activate to ignore fatigue for 1 minute. (There's a ring from Plaguestone, but depends on the DM to let you have one of those.)
There's a big roleplay issue I see with playing a barbarian. You try to be bold, brash, rash even. You want to rush to the next battle. Suddenly you're the one holding the party up. 'hold on guys, don't run so fast. Let's take our time and not get tired!' Maybe they can hustle longer than others, but there are a lot of times that won't matter, and also other ways to get fatigued. Fatigued seems to be something I've seen a couple DMs just drop by fiat, not by party choice/bad decisions. The whole party decides to chase someone (A very Barbarian thing to do) and then when you catch up the DM says you're fatigued and you're stuck basically playing a level behind or more. I don't think I've ever been in a game as dry as saying, "and then we hustled for 39 minutes, then rested". Usually they're a bit more fluid than that, but maybe DMs should be warned to not punish one player with a condition by accident. As a DM I've dropped it on my party and never really realized how onerous it must have been for the barbarian. I feel bad about that, and will try to avoid the condition in the future (unless they reaaaally ask for it).
Nobody else gets all their toys taken away so easily, and with nothing the player can do to get them back. I've seen house rules that allow rage, but halve every bonus, or ones that give you additional conditions when rage ends. But I think a simple consumable, a talisman to ignore it for a minute for instance, would be a really easy solution and basically become required kit for the Barbarian.