I'm afraid you might have missed the opportunity then. He should offer you whenever it's sunset (the sky is orange). There's two opportunities, once after the slug den quest, and once after the mine quest.
If you have any old saves to go back and witness these scenes, feel free to. But ultimately the only thing you missed out on are some relationship points and character backstory / exposition. No lewd scenes or anything.