I really liked the music in this game. It really fits with the game. And the sounds match quite nicely as well.
Movement, as you say, is really slippery and very difficult to control. I could suggest using an animation curve to map how the acceleration behaves, and tweak linear drag to control deceleration rate.
Knocking objects to take out mice feels pretty fun, and thematically appropriate, but there are way too few objects for it to be that meaningful. So I'd suggest adding many more objects, since there seems to lie the core gameplay loop of this game (You jump, maneuver, and push objects to take out mice). Because, otherwise, it's just a pretty bare-bones platformer in which you run around. (No disrespect intended)
So... yeah, this has potential as a simple, nice minigame. Just lean further into the "knock over" verb, and this could turn into something pretty fun.
Side-note: I really love the cat's sprite. It's animated beautifully!