Hello-
My group is starting red hand of doom soon, and i thought i'd at least post up my thoughts if anyone wants to chime in feedback-wise.
I have 5 players at level 5: witch, investigator, swashbuckler, barbarian, cleric
My current plan is to run the intial encounter as written just with hobgoblin soliders (level 1), hobgoblin archer (level 4) and an adjusted statblock of archer for the cleric
plus hell hounds.
if it's too much I can decrease the second wave or omit it entirely.
My other thought down the line is using a Siabrae for the Ghostlord.
I don't know how much time you have to convert or what you're aiming for exactly, but you have a lot of ways you can go about this! I'm not super familiar with Red Hand of Doom, but it looks like that encounter would pretty much be the only one in the day - something to alert the players that something is up. When converting, I like to think about what this encounter would mean in the PF2 encounter sense - and I would likely keep it around a Severe 5 encounter. With your group of 5, that means you have an XP budget of 150 XP. The second wave, though, throws things off a bit - as you said, you can use it as stretch material to see if the PCs are handling the encounter before throwing them in. As written, though, four rounds is way too long in PF2. The entire combat should be over by then, unless the hobgoblins are playing super defensively, which I don't think is their goal.
Now let's look at a one-to-one conversion like you've laid out:
Six hobgoblin warriors (level 1) are only 10 XP each to this party and are going to have a hard time posing a threat. They will very likely go down in a single attack from any of the martials or a good AoE from the witch or cleric (depending on their spells). I'm not sure that gets across the feeling of "hey, these guys are a threat" effectively.
Two hellhounds (Level 3) are a bit better in terms of lackeys, and they have somewhat reliable damage in the form of their breath weapon. That said, it gives a chance for players to feel good by dodging the flames, as DC 19 Reflex shouldn't be too difficult for the group, especially if you stick to the spread out battlefield like the module suggests.
War Cleric will likely want one of two things, depending on your time/desire. One create a monster using the rules for Building Creatures to make something that feels more in line with what you need. Or two, find an NPC that's close and reskin it - the Priest of Pharasma might be a good fit here, just give it a new weapon, maybe even change some spells around and you're good to go.
The Bladebearer is a bit trickier, depending on what you want to do. Again, the rules for Building Creatures is a good way to go if you want to make something super fitting (a dual weapon wielding agile dervish), or you can settle for another NPC and reskin it. This one is a bit tougher as I don't know of anything off-hand that gets you there so close. You could be much more loose with the idea and use a Hellknight Armiger.
What I would do:
So first, each enemy in the combat plays a role and the overall feeling that players should get is that the hobgoblins are organized and dangerous. The rank and file soldiers have their job firing from treeline (looking to use Take Cover and Hide in conjunction with their ranged attacks to be able to land hits). The bladebearer and war priest are the leadership and look to keep the PCs occupied while the soliders get in their blows. The fight should be focused on these two. The hellhounds are there to assist with flanking and skirmish with backline characters.
With that in mind, I would technically increase the XP budget to let us utilize the second wave more and let us use a lot more opponents than the 150 XP budget would allow. I'll give us a whopping 210 XP budget and putting us (technically) in the realm of Extreme 5. Yikes. That said, with such a large number of low-level creatures, I don't feel like the fight will be quite that difficult.
First things first, I'll make the hobgoblin soldiers elite and drop their number (in the initial wave) down to four. That eats 60 XP of our budget and gives players a bit more of a challenge (even if their Stealth isn’t fantastic). From there, I’ll keep the two hell hounds as is - another 40 XP out of the budget and we haven’t really started to threaten the PCs yet. I’ll use the Hellknight Armiger as our Bladebearer stand-in (though, if I’m being honest, I would much rather recreate this NPC as they will be showing up later) and a Weak Priest of Pharasma with a few more harm spells and maybe even a paralyze thrown in (perhaps we can believe it’s a granted domain spell). To keep them in melee a bit, I’d bump up the AC by 2 (back to normal levels), give them a shield and a morningstar, and call it a day. All this totals to 170 XP - one hell of a fight. Then, if things are going well for the PCs, four more regular hobgoblin soldiers come in after the second round of combat to lend their aid.
Remember, numbers were quite different in 3.5 and the goal of the game there was often death through attrition - or the very least getting the party to spend their resources. If you go forward with this conversion, don’t be afraid to make big changes to things!