I think it needs a bit more polish (like you've said). It would be cool to see a sort of linear increase in speed on the paddles that translated to a linear increase song speed as well. Cool concept!
Great work!