I genuinely think this is a great idea and wish I'd thought of it. I agree that the way it is does slow it down just enough to be annoying.
I'm glad I could help. If you have any feedback for my game it would be appreciated as well - this is my first jam so looking to hone my game design skills.