This FAQ might help:
Core Rulebook FAQ wrote:
Ranged Touch Attack Spells and AOOs: When you cast a spell that allows you to make a ranged touch attack (such as scorching ray), and an enemy is within reach, do you provoke two attacks of opportunity?
Yes, you provoke two attacks of opportunity: one for casting the spell and one for making a ranged attack, since these are two separate events.
(Note that at spell that fires multiple simultaneous rays, such as scorching ray, only provokes one AOO for making the ranged attack instead of one AOO for each ranged attack. It still provokes for casting the spell.
This answer originally appeared in the 9/11/12 Paizo blog.
This FAQ deals with a regularly cast spell mind you, but we can still extrapolate your answer.
Since Casting a Swift spell doesn't provoke an AoO for the casting, but provoke for the ranged touch attack...thus you only have 1 occasion (instead of 2) of having your concentration broken and your spell fizzle out as the ranged touch attack is part of the spell casting.
prd wrote:
Ranged Touch Spells in Combat: Some spells allow you to make a ranged touch attack as part of the casting of the spell. These attacks are made as part of the spell and do not require a separate action. Ranged touch attacks provoke an attack of opportunity, even if the spell that causes the attacks was cast defensively. Unless otherwise noted, ranged touch attacks cannot be held until a later turn.
