I did indeed notice that the game sadly shipped with a bug (as it's supposed to in a game jam :') ). After testing it the game is still playable, and in endless it's even possible to survive with everything going out of sync as an additional challenge ;).
I appreciate the feedback and will upload a fix for the sync issue once that's allowed again. Thanks for playing!