An underground group of "heretics" contact the PCs. They want to help but can they be trusted? These heretics have been "proven" to have committed numerous crimes using the church's name.
The heretics is small, maybe less than 10, consist mainly of commoners so even if they can be trusted, how much can they contribute? They might follow the PCs plan if it is not obviously suicidal.
If they cannot be trusted, they might be a group of commoners thinking they are fighting to "purify" the church but is actually led by a demon. The group is a honey trap to identify problematic elements. This can lead to the PCs being led into a trap if they are newbies or they can turn it around by unmasking and killing off the demon.
Later when they nearly defeat the demons and learn that actually they took over to fight off a devil incusion whose leader is
i) fallen paladin
ii) unknown, they are trying to find out using the honeypot tactic
iii) the town mayor
The fallen paladin may be
i) unaware of the devil incursion
ii) scout for the devils
iii) trying to rally the town against the devils but found demons instead.
So the demons parley to fight off the devils together, after which the demons promise to leave peacefully. Newbies might fall for this. Laugh at them when the chaotic demons start sneak attacking the party near the end of the battle with the devils. Party will have to solve this problem on their own.
For further expansion, the town can either be rendered inhabitable and require adventurers to protect them as they slowly crawl with wagons of their belongings to the next town, or be devastated but remain livable. In the latter case, the party may be required to help defend the town, secure resources or even be mayor.