For weapons you make a mod and add an entry to KinkyDungeonWeapons global variable, I think.
You have to put the icons in Items/ in the zip and make sure that that the name of the icon (minus ‘.png’) is the same as the item, capitalization included