I really like the concept. Though I think it could've been a bit less confusing if there was some kind of map of the dungeon. A lot of the time I was just sort-of running around until I got it to work.
Then again, the game jam just finished and I might just not be as concentrated right now.