Somewhat of a breakthrough, enabling FPS control in the ini file made mouse movement much more consistent/responsive. Vertical mouse movement still makes the character/camera wobble from side to side.
max_fps = 120
One other odd behaviour is that the camera in 3rd-person Sea periodically keeps snapping to different angles. Sometimes it snaps the angle around almost 180 degrees.