It was a bit difficult to understand how exactly this mechanic worked, but the idea is there and you should keep working on it!
Yeah its a bit weird, I probably should have gone for something more straightforward for a gamejam. But I really liked the idea and thought with a proper tutorial/start level it could have been ok.