This bug only happening in the WebGL build. Either my build settings are wrong or something is wrong with unity's export system, probably the latter(cause it plays fine sometimes).
Should I remove the WebGL build?
Anyway thanks for playing my game!