After you rescue him from the demoness in the dungeon, wait for him after you leave it and you start a scene with him, one of the options should allow you to invite him into your home during the talk.
