Try to interact with the teleport stone (that stands in the village). It's probably not activated or you haven't finished the hidden village quests yet and have no permission to return to your village until you gained Kraal's approval.
Oh thank you for the explanation. I didn't finish the Hidden Village quest yet since I'm not done increasing Nixie's Relationship (I'm not using the Tower cheats, just the Stone Teleport in there and the free shard at the top)
