Figured it out, your map is too big and enemies can't move to me for like 10 minutes
I will greatly increase enemy speed when far away of the player to solve this problem.
The enemies always spawn close to the player, because the map is infinite. While testing, I haven't any problems like you, but this problem will happen depending on player play style.
I will fix it.