I think it's hot, personally. I think it's cool if a harem member assumes control of one of the other members. Makes it kinkier :)
