He does eventually leave, however he'll instantly go away with the Brew.
If he is bugged, you can use the debug scrape command to delete him (as well as all occurring events) note that this command can cause additional issues and bugs since it doesn't cleanly disable events.
Just remember to sleep before midnight and he won't ruin your day.