Those are super good observations! yes indeed the music was intended to be uncomfortable until you matched the correct two clothing pieces together at which point it became more harmonious so depending on how one was playing one could definitely get lost in the dissonance... I feel you big time.
We were trying out the idea with modes switching since it was the theme of the game jam, but if we were to continue working on this i think it would be more effective with changing instrumentation with each clothing option and keep the actual song more similar/harmonious. I think it would align with the visual aesthetic a bit more and hopefully maintain an overall more pleasant experience for the player while still keeping the music interactive. Thanks so much for taking the time to comment on all this and happy new year!