First off; the concept is fantastic and you both should be proud at finishing a jam game
As others have said, the controls leave a lot to be desired. I was able to finish it incredibly easily, though, by just standing under red boxes, swapping view, then swapping back. I also managed to get stuck outside the map by waking out of a door while in swapped view (and was unable to get back in)
So, my advice would be, depending on how easy you find it; either ditch swapped view, have it as a map only, or save the player's height when switching between the views.
Also another cool QOL feature (that would actually work really well with the current art style) would be to show the layer ahead and behind you as faded outlines so you have a better idea where you're going. For example the layer 'E' takes you to could be orange-tinted, with diagonals going top left to bottom right, and 'Q' could be cyan-tinted with diagonals going the opposite way. You could also change the Q/E controls to W/S to make them more intuitive while allowing them to map onto the arrow keys easier. It might feel less like swapping layers, but you could maybe have to hold the key for a short time while time slows and an effect plays (which would also make the jump-swap more skillful)
Best of luck with your post-jam version if you do one!