The amulet that makes very strong atk items cause a 20% bleed doesn't work with the guns you get from the gun amulet. (Side Note: Is there a way to get bullets back?)
I'll have to check that. Currently there is no way to recover bullets, only by replacing the weapon with a new gun.