Did you get a room at the inn? That's the only way to wait until everyone is asleep, if I remember correctly.
ah thank you, don't know how I forgot that was an option.
