You build his house during the Bareshade Expansion event, so if you didn't do it yet, stop looking. Did you try meeting him on the track maybe?
I did meet him on the track, by taking a break at River's Bank, but there's no option there. I'm still in the middle of the Bareshade expansion though. I realized that might be where his house is when I went back into my game
