Early game, you are going to have to deal with rocket tag. As it stands, certain classes can be one shot by lucky rolls from >CR 1 enemies already. If you are okay with a gritty game, and by gritty, your fighter getting one shot by a kobold's lucky crit, then this might work.
Also keep in mind that a fair number of enemies are tough because of their HP. For example, cutting a caster's HP in half isn't nearly as damaging to their overall power as cutting a barbarian's HP in half.
As it is, in my experience, most combats don't last a large number of rounds. The best place to look when making combat take less time is streamlining combat rules to make turns take seconds rather than minutes.
Some things you can do:
1) Take average damage on attack rolls.
Greatswords do 7 + stuff damage instead of 2d6. A level 10 fireball does 35 damage instead of 10d6.
2) Roll for anything you want to do on your next turn before your turn before your turn happens.
If you know that you are going to attack next turn, roll your attack roll, tally up the numbers, and when your turn comes back around, just tell the DM what you did and how you rolled. Worst case, your plan doesn't pan out and you have to take your turn as normal.
3) Enforce a time limit on the taking of turns.
Don't let a player sit there for a minute thinking about their turn. An in game round is 6 seconds, so I make a rule that you have 6 seconds to make a decision about what to do. This speeds up combat significantly.
As far as grittiness, a lot can be achieved by just changing how you view HP. Think of HP as a pool of luck and stamina, rather than meat. "Hits" turn into lucky dodges, stressful parries, or slight nicks, scrapes and bruises. The hit that drops your target is the only "real" hit.
I have also toyed with having HP thresholds that Fatigue, then Exhaust characters. Then, make casters that are Fatigued or Exhausted make Concentration Checks to cast spells. This makes the game feel grittier without changing a whole lot.
For example, at half health, characters become Fatigued. Martial characters start taking a -1 to their attacks and AC. Casters now have to make checks to cast spells. At 1/4 or less the martial characters are taking -3 to attacks and AC, while the casters maybe have to make a more difficult check to cast their spells. (Dc 20+2x Spell level?)
This can also speed up combat as monsters that are fatigued/exhausted are going to get hit more often, and players that are injured are going to try to end combat ASAP, even if that means running away.
Hope this was helpful.
