Cool concept! It would be cool to see how your decisions affected the world more. Like maybe heroes come back and gift you some mementos from their quests, and your building starts filling in some of the large empty space with more and more things.
Good work, very creative idea!