naa, don't worry. you knew it was unforgiving and you broadcasted it loud and proud. there's definetely an audience for that.
i think the key to difficult games is that the game never feels random in it's outcome but rather the player knows that they messed up. so rather than making the game easier by some random amount make it so, that it is unforgiving but always straight forward and understandable.
which is super hard in VR, now that I think about it ;-) too much degrees of freedom