Yes, you can.
Every type of creature has a racial hit dice, including humanoids - who have a special rule that lets them switch 1 racial hit dice for one class hit dice if they've only got one. That's why normal characters have no racial dice.
If you want to advance solely by that, then basically you just improve them by whatever their racial characteristics are. For humanoids, this is a d8 hit dice, 3/4ths BAB progression, good Reflex saves, and 2+Int Skill Points. So if you wanted to slightly increase a humanoid, you'd basically just have them "level up" them with those characteristics. It's like normal leveling, but without class abilities and spells and such. o wo/
The jankiness tends to come in because of the disparity between Racial HD and Class HD. As a general rule of thumb, one Racial HD is less powerful than almost any Class HD (especially for Humanoids), so you'd need more Racial HD for the same CR improvement. This is somewhat intentional - they get more health and feats and such for the same CR increase this way, which makes them tougher without giving them the powers of any specific class. The further away you get from a creature's base power, though, the more likely it is there will be differences between it and other creatures of its CR.
Speaking of this, I think the Talented Bestiary is supposed to be out soonish. They're working on getting the art in, but it's basically a redesign of the monster creation system that allows you to quickly and easily scale things up and down. XD It's a third-party product, but it's made by some industry veterans who know exactly what they're doing, and it's probably going to be worth taking a look at if you have reason to change up the power of monsters a lot.
