There's 2 main reasons. First is ghost collisions - unity's collision detection system has a known issue with repeating track segments. The workarounds that i found were too complicated to pull off for me, but i'm working on them.
The second is the ramps that can show up. They look like the track, and spawn at random locations. They can be hard to see, and throw you very far. They can be frustrating, but hopefully also fun?