I had a a list of ColorGroup and ColorGroup has a list of gameObjects (or SpriteRendrer). I assigned different gameObjects to different ColorGroups and upon picking up a Color I assign all the gameObjects in that ColorGroup to have that color.
This is probably a bad solution because it is not as scaleable, and cannot be used with prefabs as easily (if you are using unity). In hindsight I would assign a script ColorGroup to each gameObject with the method "GetColorGroup" or something along those lines. And have the ColorController iterate over all the gameObjects in the scene and check if the ColorGroup is current and then change the color.
I am very happy you like the concept. Hope to make more games in the future!
Thank you for playing the game.