I think some feedback when your bullet hits something (particles, screenshake, etc.) would have made it feel a lot better. Same goes for when the player gets hit.
Ship movement was a bit stiff and unrealistic, and some basic animation would have also made it look and feel better.
The lack of an end/game over screen was also disappointing, and tracking your high score would have given me more reason to continue playing.