for me its the fact that you are walking in an endless void with no monster witch is terrifying te me. especially finding out that you are not realy alone.
Honestly it's relaxing for me, and not being alone was nice, means you have a buddy to hang out with : ) I'm sure the shadow man is very nice once you get to know him