It's not displayed very well! I'll have to work on that next time.
When dropped into player driven town scenes you may interact with townsfolk (pressing E while next to them), or head straight to the inn to progress. The inn is the south-most building (E to interact with the door). There is usually a chain starting with only one person that will talk to you, then another person will become intractable, and so on until there is no one who will respond. You may head to the inn at any time, but you could miss something if you don't exhaust townsfolk interactions.
Thanks for playing!