Not a glitch. If you hit "STOP EXPLORING" it will return you to your last location.
To go back to the "overworld" you either go to Morgana's demesne (where you initially came from)
Or if you have progressed far enough in the story you get one in your manor.