First, the link to the item in question: Holy Symbol Compartment
Does that mean, the item/potion is always considered "in hand" for the purposes of drinking the potion? I mean, a cleric can grasp the holy symbol for "free" when casting a spell (assuming a hand is free in each case), right? So, if I had a holy symbol around my neck with a potion in it, and a hand free, it would be a standard to drink the potion?
The holy symbol is a holy symbol; it acts like any other holy symbol, assumed to be available if it's worn and the cleric has a hand free. If the cleric wants to retrieve the potion vial from it, they still have to spend a move action to retrieve the vial, same as if the cleric had stored the vial in a pouch on a cord around their neck.
The benefit of the item is that it's easier to smuggle a potion vial (or an item of similar shape and size) into places where you'd normally have to relinquish your items (but not your holy symbol).