You need to wait until the fireworks festival. After that, you will be able to sleep with him to your hearts desire!
