I got it after spending time with him at the weekly fair in Bareshade. Visit his stand, then there will be an option to invite him to explore it with you.
