You need things that prevent spinning around forever being a viable tactic. You can do this by having enemies that shoot around the player for example, forcing them to stop and dodge. Bullethells also do the same by filling the screen with bullets so you need to weave around the bullets.
Nice game feel, juicy and fun.