The windup of the hammer strike is WAY longer, so it is far slower than it looks.
There are other drawbacks of the hammer, the handle doesn't damage them so you have to be more precise, and it doesn't block arrows and maybe swords too... I'm not sure.
You are correct, sir. The hammer indeed does not block swords (At least at this point, design subject to refinement, void where prohibited, must be a resident of Earth to apply.)