Sounds like you need to lower virility, where are you getting it from? Some sources can be removed, like Lucille's blessings and certain Runes.
If you mean that you want the high virility, then you have to put up with the side effects, it's not a huge weakness. It just means needing to orgasm more often which is manageable.