Love the idea, was getting into a flow despite the problems other people mentioned.
I agree with improvements suggested in the other comments. Sound, feedback etc would be a huge help. I think the randomness of the timing was the most frustrating. You made it so that mistakes aren't too punishing to make up for that, but if you did everything to some rhythm, you could make the penalties harsher and the pace more intense.