I would house rule that :
Nothing prevent you from casting a spell while prone but :
- If the spell has no somatic component, then you have no penality.
- If the spell has somatic components then you are hindered by the ground just like being hindered by armor : you get a 5% per spell level of spell failure (1st level spell : 5%, 9th level sepll : 5X9=45%)
An alternative would be a dexterity roll with a DC 5 + the level of the spell he is casting or lose the spell, to make it more dependent on the ability of the character and the complexity of the spell. But i prefer the first option as it is more simple.
To me, the concentration check is best used for something that actively perturbating the casting of a spell. An armor is not doing anything actively, neither is the ground.