You must drop your gun, then choose the attachment item, and then click on the dropped gun.
To get rid of an attachment, just throw your gun on the ground.
