itch.io

Letting the end-user load in their own files is something I thought about too - however, this is basically what Wwise lets you do already. If making music using a responsive system like this is your goal, you should def check it out. It's capable of building crazy music systems!

It's funny that you mention using the system for music. I have been thinking about this too. I would be very much interested in designing a system that integrates both ambience and music, and manages the 'vibe' of both. In my opinion, the two should always go hand in hand. Like, how can you write music for an environment if you don't know what the ambience of that environment sounds like?