From what I can see, you don't need to be a full caster, just something that gives you a cast spell ability. For example, the rogue minor magic. As long as it does not turn into a 'innate spell' or something like that.
The only rules I found was on activating a item where it talks about if it says cast a spell, you need a spell caster ability to 'cast a spell'. All of this is purely off my memory.
I'm wondering how a rogue could activate a scroll, or a fighter a wand. Is it just going MC or am I right and anything that gives you some sort of casting ability that is not just a innate spell gives access to scrolls and wands? (not staves since they have their own rules for regenerating spell points) Does this also mean that you can use a 10th level scroll at level 1? I'm sure I missed something and someone here will be able to quickly point me in the right direction.