Does your male character have the trait "Fertile"? or perhaps have you involved him in some sexual act in the position of female (your keeper got fucked by female)?
It's actually completely irrelevant.
I've had male characters with and without the fertile trait get pregnant, and also had a male character get pregnant from a female during fingering, lol. So, it seems that there is currently no limitation.
I'd love for this to be an ongoing option (a simple toggle) - but I also think that currently, it's not quite working as intended.