Post by Nikolay Bychinskiy in Out of Touch! comments

Viewing post in Out of Touch! comments

Nikolay Bychinskiy253 days ago (6 edits) (+1)

We encode videos with ffmpeg using x264 codec with constrained baseline h264 profile and if it has audio it’s encodeded with qaac codec. If there are some issues with video playback for some reason you can try unpacking movies bundles and transcoding to VP8 and then bundling again with Unity 2019.4 for windows platform but there is no hardware acceleration for VP8 in Unity (even if platform supports hardware acceleration, the only exception is android where unity will sometimes have hw acceleration for vp8 videos).

Here’s ffprobe output for one of the videos, I assume rate refers to sample rate and ffprobe has no problem reading sample rate:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '.\teresa_trailer.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.3.100
  Duration: 00:01:44.17, start: 0.000000, bitrate: 7969 kb/s
  Stream #0:0[0x1](eng): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080, 7642 kb/s, 60 fps, 60 tbr, 60k tbn (default)
    Metadata:
      handler_name    : ?Mainconcept Video Media Handler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.3.100 libx264
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 317 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]

I can provide reencoded bundle but you will need to contact me on discord and I can’t guarantee how long it will take since I don’t have any scripts prepared for VP8 encoding and last time someone needed it was too long ago and we have some new videos + some old videos have been improved. We are not going to make VP8 default because it loses in every way - it works worse on low end hardware, it either is larger or has lower quality or both. It’s not worth for platform with barely any players. VP9 and AV1 are both not supported by Unity and also are significantly more computationally expensive so even if they were supported not worth it (and neither does it allow adding codecs to built in video player, entire video player has to be reimplemented entirely to add new codec and to make it reasonably performant it has to be native plugin so it’s a massive pain).

Like Reply

elsandosgrande252 days ago (5 edits)

Hi, and thanks for the info! I knew that AV1 was heavier on the CPU, but I wasn’t aware that VP9 was too, not to mention not being able to easily include videos in multiple codecs :/

So, this is probably gonna sound a bit silly, but video playback just started working flawlessly with full hardware acceleration once I switched from Wine’s built-in DirectX(/Direct3D?) 11 implementation to DXVK — it’s even using functionality called VideoEnhance in intel_gpu_top, which I’ve only ever seen one other application do and which I still don’t fully understand — so this ended up being a bit of a wild goose chase in the end. Between that and the only seemingly relevant information that I could find online being H.264-encoded videos not working in Steam games due to Steam likely not wanting to risk stepping on any patent owners’ toes, that is not enabling H.264 on Linux by default and instead opting for a mixture of reencoding videos on their servers and whitelisting specific games (and Media Foundation apparently not being supported/implemented in past versions of Wine), I apologise for the confusion 😅

By the way, I’ve only played through the first fifteen chapters thus far — yes, I skipped over a few video scenes without even realising that they were video scenes and thought that Wine or the game was acting up — but this has been the most beautiful and immersive visual novel I’ve played on itch thus far, so, to you and the rest of the team, thank you so much for creating this amazing game and I wish you all the best!

Edit: So, I just had a thought: Instead of the built-in DirectX 11 implementation being faulty, it could be that it just defaulted to my NVIDIA dGPU instead of my Intel iGPU, which would nicely explain why video playback returned nothing when it was decoding the video on the iGPU. Now, generally, OpenGL applications default to the iGPU on my laptop — yes, it did seem to fall back to OpenGL rendering according to the terminal output, and no, I didn’t check which GPU was in use — but, seeing as Vulkan applications manage to always default to the dGPU unless I force the iGPU, I could imagine Wine’s DirectX 11 implementation following suit in spite of using OpenGL in the end.

Before somebody asks: “Why didn’t you install DXVK right away‽”, well… I did, but MinGW64 being kinda fragile and me being kinda liberal with compiler flags on my installation of Gentoo led to any and all games crashing immediately upon trying to open any sort of window because of faulty stack smashing protection getting tripped. You can see the DXVK issue ticket here, but, in short, removing -fstack-protector-strong from my global CFLAGS (which I actually did a while ago but just didn’t rebuild MinGW64 in the meantime), adding -fno-stack-protector and removing -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 for MinGW64 specifically for good measure, and rebuilding DXVK fixed the issue for me.

Like Reply

Nikolay Bychinskiy252 days ago (4 edits) (+1)

Well, I was always saying that you need Wine+DXVK for game to work properly on Linux, no DXVK always lead to some issues with videos for people. Those who don’t have vulkan capable hardware need VP8 bundles but that’s very rare, pretty much everything in the past 10 years should support vulkan.

Speaking of patents - there is a way to avoid having to pay royalties for using h264 by using prebuilt OpenH264 binaries from Cisco. Limitations are that it only supports constrained baseline profile, must be downloaded on demand instead of bundled with software and must be optional (so there should be option for user to disable OpenH264). And I’m not 100% sure on that but I think if there is hardware codec then royalties are on hardware manufacturer rather than software that uses it. Video codecs are a patent minefield but all the royalty free codecs are designed by companies that only care about web and reducing traffic so they are all extremely computationally expensive, so h264 is still the best consumer codec that supports decent quality/compression while working well on low end hardware.

Like Reply

elsandosgrande252 days ago

I see. I didn’t know that OpenH264 couldn’t be bundled, although I guess in hindsight it makes sense so that Cisco can keep track of how much they need to pay in royalties.

By the way, is the Discord server the right place to report bugs/issues with the game? I have experienced the game crashing/closing unexpectedly when not in focus, but it’s very infrequent and seemingly completely random, so I haven’t encountered it on the few times that I’ve had the game running in the terminal as opposed to the itch.io client. If the server is the right place to go regarding this, I’ll join once I have at least some idea as to what’s going on.

Like Reply

Nikolay Bychinskiy252 days ago (4 edits) (+1)

Yes, currently you can only report bugs on discord server in #oot-bug-smashers channel. Keep in mind that work on current runtime is mostly stopped and all focus is on big engine upgrade that will come out next year so I’ll only fix if you can provide enough information and it’s relatively simple. There is a good chance this is a problem with Unity itself because the only thing that game does on focus change is muting sounds if that setting is enabled and also stop handling mouse events except for scroll (so that clicks that return focus to window would not be handled by the game).

Like Reply

elsandosgrande252 days ago

All right, thanks for the info and have a nice day!

Like Reply