If you used "Time.timeScale" to speed up the game, colliders won't be able to keep up with the objects mesh speed after a value. You need to use different methods to fix it instead of scaling time if it was the issue. That could be the cause of misses. Or if you are waiting for the animation to finish to allow player to hit again, that is definitely the problem. Hit detection misses only occur when the game gets fast enough, at the beginning everything works correctly. Hope that helps.