The white cyclopus wouldn't latch if you're wearing Violet or Ice Armor.
Also it wouldn't latch on if any of the normal cyclopus is already latched onto you. So the best way is to use area spells like Meteor Shower or Maelstorm to kill off the two normal ones, leaving the white one.
(Optional) Use Charm spell to make it latch on immediately.