Update: Using the OpenGL Version works and runs perfect, still not sure what the issue was for Windows version...
I've been running the default version for windows, I'll try the OpenGL version
Not sure why but despite everything running fine, celeste seems to load the lighting weirdly. I tried reinstalling both the game and the XNA framework thing.