Believe me, you're not doing anything wrong - the difficulty of the game is still a bit too high. Although the reception has been generally positive so far (thanks!), the one common denominator among them has been that people get stuck on waves 4-8, which... tends to scare them away from playing for good, blocking them from evolving as a player.
So, considering a heavy rebalance is due, I've nerfed enemy spawn rates by a little over 20% after reading your comment. Hopefully the game will be a bit more forgiving now!