I've the same problem..couldn't remove Raina from party
Make sure you guys are on the very latest version of Harem Collector, it will get rid of the bug. You can switch Raina out of the party. I know this because I did the same thing