This is a feature! You can only turn in one direction or stop, not go back. It should make it a bit more of a challange.
I planned to add more guns (and a flaming sword on one side) to the other sides with updates, to make it easier, while enemies come in faster and stronger. But did not get this far.