He leaves after you advance Elly's event to the point where you talk with him for a potion. Afterwards, he is gone forever and doesn't appear at any other location.
The butler should give you the crown in an automatically started event when you walk in or out of the main entrance of the castle of Aldlyn (the exact spot is 2 tiles below the big gate) after you did advance the main story and repaired the broken barracks after saving Mira.
