Raistlin Majere, often described as sickly and weak, had a CON of 10 in most “official” stats IIRC.
So this points out something to consider: an author can have their character survive all sort of deadly encounters simply due to the power of Plot. An ROG character, with rules and dice to decide the outcome, could be dead many times over. So don’t use what a character accomplished in fictional novels as justification to run a concept.
I think I it’s pointless to run a CON 3 character. Sorry, try all the clever builds you want, you are going to be running far behind on hp and will always be dealing with abysmal Fort saves. Plus 3 points of CON damage and your character is dead.