Esc > Settings > NPCs Respect Sexual Orientation.
Without ticking that first (basically equivalent to wearing a sign stating your sexual orientation), just because you're only into males doesn't stop women attracted to you from hitting on you cause they wouldn't know.
