Really interesting concept! The driving was a little dizzying though, cinemachine has some easy to use cameras that can smooth out that type of motion, maybe that could help.
It was due to the 'reusable prefab' workflow I made to save time, the camera is following in update whereas the car is moving in fixed update, perfect recipe for stutter :/