I liked the idea the graphics are cute! But as others said, the colors are too fast in the beginning. Beyond making it a little slower, I think having some kind of constant visualization of the relation of colors/arrows would be nice. This way I could see that up is blue for example. As it's already kind of difficult as the sequences increase, an aid like this could help the game flow.
Also, it may be interesting to include some background to make it less plain.