No, your game is not bugged. The only way to get there is by Darja taking you. If she didn't take you then it's because you already have a lair. You can have the Ancient Spire, Sunken Temple or the Old Mine. Each lair has their own unique ally. You can't get them all at the same time.