This is a really fascinating concept, sadly I couldn't seem to get it working with my Rift S in either oculus or steam VR. I would only be able to load in without any hands or inputs through steam... strange.
Either way, it sounds like an exciting project so hope you continue on with it!