Thanks for the comment :)
The slow down when the background turns white may be due to the shader I wrote that inverts the colours, however it's quite simple and lightweight so I don't see why it would slow the game down... I made and tested the game on a pretty old laptop too?
As for difficulty, I'm afraid I didn't have any time spare to ramp it up over time and also test that it doesn't become unplayable.
After the Jam is over I might consider updating it a bit and releasing a 'post-jam' version though :)