Thanks for all the thoughts so far!
The language is dead-dead, several languages removed from anything spoken currently, so it isn't possible to see pieces of it in modern speech/writing.
I have a couple of reasons for wanting to include the language. It will serve as a means to provide the players with information about the nature of their world. It may grant access to certain otherwise inaccessible magical items (such as ancient scrolls, etc.) It will also possibly serve to feed them plot information.
I also like the idea that earlier bits of the language they encounter and cannot understand at all at first could later be revealed to be interesting and/or important.
I have been playing with the idea of "other things in their downtime" sort of. In other words, just giving one player the extra skill and giving other players other unique perks. There are seven players total though, so it will depend on if I manage to come up with six more interesting perks. :p
I hadn't considered memory crystals. That could be a great way of introducing the ability to speak it, which couldn't otherwise happen without a lot of exposure otherwise.