The Swashbuckler already has their Precise Strike adding additional damage to all their agile or finesse weapon strikes while they have Panache in effect. This shouldn't be dismissed because it's effectively more damage than if they only added dexterity to damage as it's strength + precise damage which will increase as you do. There will be very few instances where you won't have Panache while in combat so you'll most likely always have it ongoing. Plus using the Finishers are just cool conceptually so losing the Panache to use one isn't a big deal.
Dexterity has its uses for AC, Reflex saves, skills and finesse weapons making it overall a good defensive and utility stat. One of the things I enjoy about 2E is that every stat has a defined purpose to warrant their use, and having mechanics where you can pigeonhole everything you need to do into a single stat makes for a rather unhealthy system as it removes any mechanical risk to the game by removing strategic stat distribution. And with the number of Ability Boosts you receive every 5 levels it helps strengthen your character where you'd want to be through natural progression.
Dex to damage should remain an exclusively Thief Rogue mechanic anyway because that's their niche ability. Speaking for myself, I feel like it's the least interesting rogue racket because of it. Honestly, ruffian and scoundrel just seem more interesting both mechanically and conceptually because of their emphasis on other ability scores to do more interesting things.