Fixed the jitters! My camera movement was in LateUpdate already, but it used a lerp function. Moved it to fixedupdate because that's where my player movement was happening and it got rid of the jitter :)
Well done!
