it seems the approach move route is still triggering it hmm...
I also got lucky and he never entered the room with the 3 closets until i left the closet
maybe you could hide the back door key to the cellar so you could still keep the minotaur section.
if you're going with that then a note somewhere about the trap door be easier to access, (like in place of the back door key maybe?) since as far as i can remember the only note talking about the trap door was in the attic. altho i could be dumb and forgetting if somewhere else had one lol.