If you choose that you are attracted to "anyone with a pretty smile" then the ROs will be mixed gender but in that same choice display you can choose for them to be all male or all female based on preference :)
What the person above said is correct, and to clarify: no, it's not possible to select specific genders for each specific character. It's solely randomized if you choose that your Crown is attracted to both men and women. That's an intentional design choice on my part!
