Yeah, so... there really is no good fix for that. The only problem I found that could be raleted is your mouse. You probably have one of those "high DPI-high polling rate-gaming mouse". Which is good for games but bad for development. Because the poll rate is so high it just clogs events stack. Anyway, I'll try to do something about it but so far the only way is to try changing your mouse to something with a sane number of DPI (like an office mouse) and poll rate and check if it helps.
There is another probable culprit and that is a gamepad. Or rather the lack of it. Try pluging in a gamepad if you have one.