Do you think you could have some sort of "despawn portal" that once the player crosses, it despawns a group of enemies? I know there is something like that in the hammer editor but I don't know how well something like that could be implemented to your game, though if it does start to annoy you that something like that is left in you can perhaps start there
Viewing post in Quality of Life small update comments
It's not only the fact of despawnig but the players immersion. Let's say one of the enemies decides to follow you through the map then you hit one of these despawn checkpoints while one is on the screen it would be kinda funky to look at.
But don't get me wrong its definitely something I'll have to look into because this is something that will also effect "Rupture" if I don't study into it a bit more.