First thing to note- you can get a bloodline now without using the fiendish heritage feat. It was errata'd that way. Now, the fiendish heritage just pretty much gives you three rolls to get something good from that table of variant abilities (which range from greatthings like +2 to a stat or enlarging your limbs to let you use larger sized weapons, down to silly things like just living off of ash).
Now, the bloodlines: I am not entirely sure what you want when you say 'goat' infront of demon or devil. Demon bloodline tieflings are pitborn (+2 str/cha, -2 int) and devil ones are hellspawn (dwarf stats, basically).
If you want your tiefling to look goatish... well, that is between you and your GM, since it is mostly a flavor thing. For flavor advice, the best I can give you is two "goat demons" off the top of my head. The first is the brimorak, which is small and fire focused, while the other is the Schir, which is disease focused. Hope that helps.
ohh and can you pass me a link to that errata or is it under the feat now?
Maybe 'errata' is the wrong term. "Quietly written out by every practice that followed the original release" might be better.
The basis here is that Paizo first released fiendish heritage in The Council of Thieves stuff (which was 3.5 material). it included the bloodlines, and all of those various charts.
Later, the company released another work under its own game system. This work, 'blood of fiends', which is one of their books focused on a single race and providing new options and lore. Generally speaking, such works tend to be rather extensive, and while they might not include all past options, they are internally consistent (so you can play with the race's stuff without needing other books). The interesting bit ehre is that, while the bloodlines were republished, fiendish heritage has not been. Which is where you begin to see indications that the feat was no longer necessary.
This is further reinforced by the fact that, back when tieflings were PFS legal, the bloodlines were PFS legal, but the fiendish heritage feat was not legal. So you could use the bloodlines, but you were actually barred from using the fiendish heritage feat.
So while I cannot immediately find a direct dev source on this, the publishing history is well documented in the various threads on this subject int the Rules forum. So direct your table there if you get any pushback.
Your case might be better if you direct them to the materials on assimar and dhampirs as well. Those have the same general 'bloodline' systems, but they don't have any kind of heritage feats. The fiendish heritage feat has generally been made obsolete through Pazio's publishing history and developing practices (I mean... spending your first and possibly only feat just to get slightly different stats? Not a great trade).
Anyway, if you are really worried about your table preventing a bit of minor flavor like 'I look like a goat man', then maybe you should go with the rakshasa based bloodline (beastbrood). Rakshasa each have an animal based theme (kind of like spirit animals, only you end up with a tiger head?), and as a result their tieflings end up with animal like features. So why can't your dad be a goat man? Not sure how well the art supports what you want, but this seems like arguing room.