This sounds like a bug - does the camera move even when keeping the mouse still? And what kind of mac do you have?
I have a macbook pro 2015, running macOS Mojave! When I move the mouse in a direction and then stop moving it, the camera continues to drift in that direction. This means it's basically always moving; even when I move the mouse in the opposite direction to get it to stop, it starts drifting that way instead. Hope this helps!!