I dealt with this by basically having the party help Chivane defeat the trogs, get the Mantis Blade, and send her on her way leaving Saventh Yhi with it.
And yes we played this with good-aligned PC’s and the way we justified such an act was that: 1. It kept the PC’s faction (Pathfinders) safe from assassins, 2. Part of the deal was also that Chivane had to agree to free Sasha (who the PC’s had gotten quite fond of) from any commitment or association with, or any fear of reprisal from, the Red Mantis faction, 3. The party sent a warning out to the rest of the world, via the Pathfinders and Free Captains, that a dangerous evil artifact was discovered and in the possession of a Red Mantis assassin, including her name and description etc. So it was kind of a mentality of “let the rest of the world deal with it, hey we can’t do EVERYTHING!”
And yes, that’s a tough moral dilemma situation no matter how your party deals with it. So what I then did was come up with a whole separate session where it was dealt with by characters from previous campaigns. 3 different adventuring groups, 1 acting on info from a Pathfinder, 1 acting on info from Free Captain of the Shackles, and 1 who had no info about the Mantis Blade but were hunting a Red Mantis assassin and pursued him all the way to the stronghold where Chivane ended up. The 3 parties then joined forces, defeated Chivane and her allies, and recovered the Mantis Blade.
Then the big twist, if you will, was I dropped them the hint about Mordenkainen’s Disjunction as a way to deal with the sword, and that the best chances for that spell’s success was to go to the place the sword was made and cast the spell there. In other words, after all the party’s efforts in Saventh Yhi to deal with this thing and get rid of it, and the moral dilemma of the whole thing, the players then learn that the thing is going to be BROUGHT BACK TO SAVENTH YHI for them to have to deal with AGAIN! :)
And yeah, that’s player knowledge, not character knowledge, and timeline-wise, it wouldn’t be back until the events of the AP are concluded, but it at least let the players know the situation was handled, for now, and they didn’t have to lose sleep imagining all the murders and assassinations they just let loose on the world.