I think you should get the XP for every victim, and then some more for the house!
One important part is that not that many inhabitants will take something like arson lying down. Unless they're asleep, that is...
Burning things rocks! Burning things rock!