Real interesting gimmick! Very hard little game, but interesting nonetheless.
I would suggest, as a way to make the game not as frustrating for the player (or as an option, mayhaps), to make it so you know where the player is, or just give slight hints of it, so you don’t feel as lost in the dark.
Instead of making the player constantly face difficult challenges, you can make them repeat tasks, which makes the player feel like he is in a bit more control of the game at times. That is my suggestion though, being quite the casual myself hahaha