The broken beer bottle seems a bit wonky if you're going for a serious campaign, I agree. I don't think it would be out of line to gently suggest something else.
I don't really see what's wrong with the little girl with an axe (sounds kind of bad-ass to me), or how forcing the PC to use a sword cane instead would be any better.
As far as I can tell, neither are really breaking the gothic-horror setting. I agree with Hogarth, that's one badass little girl. Axes make awesome weapons in a horror setting as well. She may not be Lizzie Borden, but violent children using a weapon meant for hewing wood seems like it wouldn't take much to turn creepy.
As for the mad friar, what's the roleplay reason behind the busted bottle? If there is none, and he just likes the image, oh well, let it slide. The PC enjoys it and so long as he's not trying to get slapstick about it, it really doesn't hurt the setting. Like the axe though, it's a slashing weapon, and unlike the axe, it's all jagged. You could make some very unnerving descriptions of the physical wounds a weapon like that would inflict. Viscera falling out, flaps of flesh, gushing arteries; improvised glass weapons are just plain mean. The character may start to rethink how he uses the weapon. If he doesn't, and seems to actually be enjoying the savagery of the broken bottle, that has other, equally awesome implications in a horror setting. Awesome from a story-telling perspective at least. The stabbing victims, not onboard with being stabbed, suprisingly.
Ultimately, these seem like minor fluff choices the players made to enjoy their PCs a little more. That's a part of the game, and a good part of the game IMO. I like the players finding ways to be involved in their characters, even if it's a little bit silly. If you're determined to maintain the mood, use the fluff they've given you rather than asking them to repurpose to suit your vision. Using the personal touches to their characters as part of the story deepens their involvement and, in my experience, almost always improves everyone's experience. I really have a blast turning barely considered aspects of characters back on them in horror games.
The girl with the axe sounds fine, but I would probably veto the broken beer bottle, which is pretty dumb. Maybe for a cleric following the god of beer, but I don't see how Pharasma would approve. It sounds like lousy roleplaying to me, absent some amazing justification you haven't shared with us.