This works pretty well in the beginning. But inside the tunnel it is too dark to give the ControlNet enough info for a consistent animation. I will try again with another sequence inside the tunnel, after the camera adapted to the darkness.
Here's what the source material looks like: http://twitter.com/finefingames/status/1388535835985948681?cxt=HHwWksC93Y3_iMUmA...
And here you can download a 614 frame sequence of this clip as 512x512px PNGs and try for yourself: http://finefin.com/tmp/TunnelSequence-614frames-512x512.zip