Heya! If I remember this correctly, it's optional to do that. You can move into the cave zone to the left of the ruined castle after you finish the first quest with Leary in the forest. There's a small road tile leading to the interactive mountain zone that will lead you to Gabby's lair.