An edit is not enough, so I'm commenting my own post...
So I finally figured it out lol Previously I thought it was a bug that the screen kept changing, because incidentally I always pressed a movement button when the scene switched. (One bug remains, though: The character disappeared somehow in my first try.)
The concept is cool and unique, but a quick introduction tutorial would add HUGE value to the game! Just a simple one where you see a speech bubble above the character's head that says "seems like the world is shifting" or something similar. I now read it in the description on the game's page, but in-game would of course be much nicer. Anyhow, well done for the time constraint!