I like the concept! I think the movement is a little too bugged. Pressing the key slightly will move you quite a bit, and it's hard to move around when you jump. It might also be good to rearrange some enemies so that they aren't right next to you when the camera moves.
Otherwise, cool mechanic and nice job!