The grappling rules state:
Humanoid creatures without two free hands attempting to grapple a foe take a –4 penalty on the combat maneuver roll.
There is a discussion about whether or not the penalty applies only when you make the combat maneuver roll, or if applies to the entire round. Basically making the question, if you do something like draw a weapon during the round you are grappling should that penalty apply even if you've already made the grapple check?
Arguing strictly from common sense rather than written rules.
Yes.
I can't see a scenario where you're holding on to a struggling combatant that wouldn't diminish your capacity to fight.