You can try to look at Normcore. It's what I used for Yogalicious. It makes it pretty easy to do multiplayer (as long as you keep it simple) and they have a great support community on discord! It should work well for physics VR sandboxes (did I just box your game in a genre? Sorry if untrue)