For me, it felt like the sfx and the music would be fighting for control, usually towards the start of the waves. You actually listed the issue yourself, so you're already ahead of me there.
Thanks for the reply. It can be tough to pinpoint things.