Yeah, that would be extremely annoying. Not sure what I could suggest other than try another browser, if you're able to.
I'm still wondering how to do movement, whether it might be best to use keyboard controls or do something with the mouse / finger presses. If issues like this are common, maybe mouse clicks / finger taps would be best.