Personally, I would allow this as a swift action and require at least one free hand.
Goggles, smoked would be great to have at all times versus Glitterdust, especially for casters who aren't about to use spells that require to-hit rolls. This would include most control spells.