Don't get me wrong, I enjoyed this game. There's some good stuff going on in there and the one boss I managed to beat was fun and engaging. I just couldn't figure out where I was going. Now that you're explained the map, that's also interesting and would probably be an interesting puzzle for a game that I would invest more time in or a competitive game ( imagine the advantage if one player figured out the map first ). It was just a lot to wrap my brain around for a jam game.
I hope you do something with this, I'd really like to see this get developed further. I'll definitely revisit it.