There is indeed a bug, thanks for reporting it. It's not exclusive to the Vengeful Talisman: if you equip a talisman, save and reload, you can't unequip it afterward.
Problem is: I can't submit bugfixes during the jam's voting period. But in the meantime, if it really hinders your progression, I can upload the patched version on Google Drive.