OOOOOOOOOOH, I get what I did wrong. So I would always click ALL of the keys on level 1 for fun and didn't realise that it carries over to other levels. So 80% of the keys were immediately red on level 3 for me, oops.
And I didn't notice the keys were changing because almost all of them were degrading for me :P
