An awesome concept! I think with some more refined movement this game could really shine.
Tip: You can prevent the SteamVR plugin from loading by default in Unreal!
Plugins -> SteamVR -> Enabled or the EnabledByDefault key in [Unreal install dir]\Engine\Plugins\Runtime\Steam\SteamVR\SteamVR.uplugin.
Unreal 5 does this by default.