I do think there's a problem here, but that problem is that restrictions are applying to the entire explore step when they should be applying only to each step within the explore step. (I have to research this, but I'm pretty sure it wasn't always broken...)
Per step restrictions are absolutely supposed to apply to the Advance the Hour, Give a Card, Move, Close Your Location, and End Your Turn steps.
This was the comment that I took my understanding of steps from. My thinking was that anytime I stared a new step or check, the restriction reset. So, when I started a step of the turn, the restriction was reset. When I started a new step of an encounter, the restriction was reset. And when I started a new check, the restriction was reset.