No, you're supposed to be able to make him stay without dating him. If you can't, well, maybe you found a bug! Let me know :)
Every time I talk to him I just get the date request. I waited until pretty late to finish the quest with the original fruit stand girl if that's what caused this.
Hm, sounds like a loophole! I will take a look at it tomorrow :)