Ok, I couldn't find anything in the CRB. However, someone thought they had read about a limit to the number you could use at a time.
Maybe they had read Vance books. In one of the novels, a guy with an enormous quantity of ioun stones did lose his memory.
Hey, if the BBEG is spending attacks swatting stones instead of murdering me, I can live with that. Keep the most expensive one in a wayfinder and/or implant any stones you're worried about losing.
Just make sure you implant the Clear Spindle last. Since IIRC, it indirectly prevents the implantation of additional ioun stones.