It was happening even in the tutorial area, but became more pronounced in the transition and then at the boss fight. So It may be that I am lagging or maybe my mouse is too sensitive and it jitters, not sure, again it could just be a skill issue or lack of sleep.
Litterally got 3 hours of sleep last night. XD