You'd be chipping away very very slowly, and with this system, no matter how high your stats you can always roll zero successes while the opponent can always roll one at worst, and they also have other options. I'd be surprised if there was a problem unless you pitted a character with 4 grace against another with 1 grace, and the latter insisted on a straight fight instead of playing to their own strengths.
Edit: and if playtesting reveals a serious problem, then I can always borrow a page from my other system and grant characters an extra die on their next roll after they've taken damage, or another way to balance things out.
