If I am not mistaken, this is the dinner you're invited to after saving her, correct?
You need to be just outside her house at around 6PM as indicated in the quest log. Can't go straight in without the key, so the event trigger itself is what you'll need to look for - you'll know it is ready if you see a drop down with the quest name coming down from the top of the screen.
If you're having trouble being around at the right time, you can try to take shorter power naps at home - significantly more time passes if you're trying to sleep off a massive energy deficit at once, so there's still some use in going to bed earlier if you want less time to pass.
And keep in mind there are some buttons around the top left of the screen for advancing time automatically, by 15 game minutes, or a game hour, and so forth. So if you end up showing up at 5PM or so you can just push up at 15 minute increments until the invite drops down.