That happens if you have a romance less than 75 at that point (3/4 of the bar) - automatic break up. You also get romance boosts from inviting them to your events, picking certain options during group scenes, giving the right present etc. The dates alone aren't always enough <3