Nice idea. It does need either a sound or visual cue that the timer started as it would sometimes feel like its frozen while it loads and cost you a second which for this type of game is really important.
That's something I didn't think of, thanks for the idea!