So...to flip things around, as food for thought:
Say your party is waltzing along, being generally observant, but unaware of any potential threats, and suddenly you get blasted by three rifle shots, taking a good chunk of damage.
Your response (and a reasonable one, I'd say) would probably be something like, "What the hell, GM?"
GM: "Oh, there were three enemies there, and each of them had readied an action to shoot as soon as they spotted someone."
You: "But we're not even in combat!"
GM: "Doesn't matter. They readied their actions."
You'd be right to call that BS, and any GM who tried to pull something like that, I'd say, is just being an ass. PCs readying actions outside of combat breaks things in exactly the same way, so I don't see how anyone thinks it's defensible.