I only tested with my Quest, and recentering the view always moves the broom to the correct position for me.
Did you test the updated version with the vignette option to mitigate motion sickness?
Which broom did you use? The first you get rotates pretty slowly, so there might be a problem that shows only on some headsets? I'll have to investigate this.