One of the key aspects of the game is that you get to play with these ancient artifacts throughout the campaign. That's half the fun of it. The AP is designed for you to keep the shards, yes.
Perhaps your GM has other ideas about how he/she wishes to run it, however. I don't know why he/she wouldn't allow you to use them though.
That's weird, why would he keep the Shards from you.
That'd be like Gandalf telling Frodo to find Mount Doom and wait for him to bring the ring to be destroyed. :-D
Frodo: "You know, I'm right here, if I had the ring, I could totally throw it in the lava"
Gandalf: "Just chill out man! I got stuck in Ring Wraith traffic, I'm almost there, by the way, did you know there's a war going on!"