There does not seem to be a definition for "enemy" anywhere in PF rules, despite a multitude of occurrences in rule descriptions. Some might say that the answer is so obvious that it doesn't need explaining. At first glance I agree, however a situation came up during play that has led me to ask what defines an enemy?
Is a creature an enemy only in the mind of a PC?
Can a creature be an enemy of a PC, if the PC is unaware of the creature's presence?
I proffer this definition based on a boolean OR truth table.
If either the PC believes the creature is an enemy, or the creature believes the PC is an enemy, then the creature is an enemy. If both believe the creature is an enemy, then the creature is an enemy. If neither believe the creature is an enemy, then the creature is not an enemy.
What does this matter?
The word enemy is used to discriminate targets for spells. Since most spells that target enemies cause harm, you might still ask what does it matter, since you probably wouldn't want to cast it against a party member.
If an enemy is defined only as "what the PC believes", then any creature (within range) that a PC in unaware of cannot be an enemy for the purposes of a spell's targeting. This definition as been posed by reason that magic works based off of the caster's intent. If the caster is unaware of a creature, he cannot direct magical energies at it.
I believe this definition poses obvious problems. Extrapolated for fireballs: Fireballs target all creatures. By rationale of what defines an enemy (caster must be aware in order to direct magic against something), if the caster is unaware of a creature within range, then a fireball that targets all creatures would not be a valid target.
That doesn't seem right to me.
Furthermore, if that formula for determining what is an enemy were applied to melee combat, then AOO's would be defunct. All a fighter PC would have to do is: on his turn declare the orc to be an ally, in order to move past without provoking an AOO from the orc. Then once in the square he wants to be, the fighter PC then attacks the orc or another creature within reach. On the orc's turn, it could effectively do the same thing to the fighter PC.
It has been argued that you do not have to be an enemy to provoke AOO's. The rules are a little unclear in this regard. They start out using the generic term combatant with regards to threatened squares, but then use enemy when talking about provoking an AOO.