Seems like playing the file back on the headset on local storage doesn't have any issues compared to streaming through XBVR.
