Interesting idea, I like it! It fits the theme well, but I do have one problem. For 99% of people, using the mouse with arrow keys is really awkward, most people use the mouse and WASD.
Want to +1 that arrows + mouse was the issue for me too though I still had a lot of fun. For a game with few controls like this I would just do both WASD and arrows and then people can use whichever they prefer. There are people with left-handed mice that would appreciate it!