you have to wait and keep visiting him weekly, at some point it will triggers after being with him in the morning
also i made a cg guide, if you want here's the link: https://docs.google.com/document/d/1xKhE31i0TlpRMTIlMNNtPZ3Iu2E11KtuMTjO7ukU2XM/... ♡