Ah nvm, I figured it out. For anyone else who has this issue, it's click and release, then move the mouse, not click and drag the mouse while holding down the mouse button XD
Maybe I should change it to click & drag
It's what everyone seems to expect