My table has always played -- since after Chapter 2 of Rise -- that you can banish as many as you want after a scenario and then rebuild as if you didn't have enough cards. It makes the game a heck of a lot more fun. And, I'd expect, especially back when you actually had to work to remove basic and elite cards from the offerings, so they'd keep popping up in decks and you'd never get to see what you wanted. (I had nothing AGAINST that -- I liked that mechanic... though we'd massage it a little when integrating class decks into the pool and remove them a little pre-emptively compared to the core adventure decks.)
There's no reason NOT to play with it. I'm baffled by your rationale not to make it a core mechanic. The "rebuild" rules of lower cards is not that egregious to power you up that significantly. And I couldn't imagine having been able to finish Wrath (*gag*) without it.