Now that the Giantslayer AP is done... the Bad Thing that happens with the Giants? Basically that the giants fail. They invade and in time are taken out by the Worldwound or the Runelords or worse. (And when you think of it, an army of giants would probably be easy prey for a bunch of Rune Giants.)
I'm thinking Karzoug and his armies would just love to setup camp in a flying castle. I think the giant leadership from Giantslayer would be quickly swallowed up by the bigger fish running amuck in these "all bad things" scenarios. Though a parting "thanks for building that handy giant army for me, muahahhaha" would not be out of line for whomever (or whatever) takes over the flying cloud castle.