Cool concept, a bit hard to grasp. Maybe you should dilute down the difficulty by adding more simple levels at the begining, people playing your game will be less good at it than you, and might get frustrated fast. This is especially true during a game jam.
The music was kinda annoying after a while ^^'