The main idea is great, you've made some interesting puzzles, and I can imagine many more with this mechanic. But the game really is incredibly finicky. I think making jump height scale less aggressively with size would help a great deal with this; as it is, you often have only one shot at a particular jump.
More checkpoints would also be appreciated (do I really have to do room 2 every time I want to take a shot at room 3?)