The door for the tower itself?
There's an illusion on it, similar to the trees concealing the underground path. You have to walk through the door on the inside in order to get it to appear on the outside.
(though tbh I'm just now realizing that's never pointed out in-game - added an event for that for v0.16)
I think you misunderstood me. Sorry if the way I worded it wasn't super clear.
The door doesn't hide itself if you miss something. It only shows up after you walk through it.
Since you can't walk through it from the outside, you need to walk through it from inside the tower.
After you've walked through it from inside, then it'll show up outside.
