Really fun concept!! I think this would be a very fun party/school computer type game!
Flesh it out, use some original work you own/can use, work on balancing the levels (maybe different difficulties? Double the time during an easy mode?) and I can see this game becoming a bit of a classic.
I can totally see this being played on something like CoolMathGames for example.