You shouldn't allow yourself to be pressured into running a game you won't enjoy, and if your players won't accept that then they aren't being fair to you.
That said, have you considered why you think you won't find this fun? The problems might not be as big as you think (Kydeem de'Morcaine's case is extreme).
I've tried explaining to them, that trying to RP any sort of interaction with normal races, is going to go down-hill fast,
Not if you make the odd races part of the world, in which case they are "normal." See for example Bas-Lag which has cactus people and scarab-beetle people, and these are mostly integrated into society and interact just fine with the human characters.
we're trying to set up an AP too(so, it's designed for them to you know, be human-like).
APs can be altered. Would it really change things that much to replace all the halfling or gnome or elf NPCs with flower-people such that the race is integrated into the campaign? Unless specific racial dynamics or xenophobia are a big part of the AP (like in Kydeem de'Morcaine's example) it shouldn't be an issue.
pretty much only one out of five has decided to make an actual race, you know, with a cultural identity and societal norms, something I could actually have walking around and met upon.
Are the players willing to create culture and societies for the new races they want to play? If so, that should not be a problem (though you will want to require the cultural details well in advance of the first session).
One of them actually asked me if he could play as a quadruped rat thing with carrion sense, and he was going to try to only scavenge to eat.
Here's where compromise would be a good idea. Explain to him that quadrupeds are normally not used as player races because of their difficulty with handling items. Would a ratfolk not satisfy this character? If he really wants carrion sense, he can trade swarming for it - carrion sense is not a great ability so it's not an OP trade. Alternatively, he might consider a rat shapeshifter.
It's also important to discuss the scavenging aspect - how prominent does he want that to be, and will it make anyone at the table uncomfortable? Are you mostly worried about what NPCs would think, or do you personally find it gross? If so, would having that activity be handled off-screen with Survival checks be acceptable?