I think one problem is, it doesn't get harder the farther you get, the reason I could rack up the score is because I could, on average, get plus two seconds in less than that time, if you made it so it slowly gave you less extra time than it would be better. It's very simple but it's a good game!
Edit: You could also have a two gamemodes/modifiers: Losing instantly from getting it wrong once and only getting X seconds per question.