It was only mentioned briefly in the tutorial, but once you destroy a house, the villager belonging to that house will exit the game (via the castle)
