That's really weird. Here's a few things to check:
1. Try adding an ignore.json file to your mod with this:
{
"Ignore_Idol_Portraits": true,
"Ignore_Files": []
}
This way you make sure that the game only uses sprites from your mod folder.
You can download the example mod from here to check how it should look like: https://sadambober.itch.io/idol-manager-editor
2. Make sure you're only changing files in your mod folder, not *game*\IM_Data\StreamingAssets\Textures\IdolPortraits