There are 2 flags for Rufus.
Start of Day 3, you get the option to go out with Rufus twice. Before Wolfstar enters Pleasure Island, agree to spend time with Rufus later. You can break off your date with Hugo and choose to go out with Rufus instead (that's the first flag down).
After the Fury fight in the church, you just need to send the other calls to voice mail until you get Rufus. Just take his call, and you check off flag 2.