Good point about clicking like a madman, there should be some reason not to do that.
Collision detection was done via List. OnCollisionEnter add to list. OnCollisionExit remove from list. OnClick, Pop and tell everything in the list to Pop also.
Thanks for the feedback!