Post by Amos in does itch app actually resume downloads?

Viewing post in does itch app actually resume downloads?

Hey, I maintain the itch desktop app.

It does actually resume downloads. Here’s what it’s supposed to look like:

However, depending on the contents of the upload, the app has a few different strategies.

For formats like .zip, which is what you get if developers use our tool for uploading games, the situation is ideal. The app actually manages to extract files while downloading - so the .zip file is never actually stored on disk, which saves on disk I/O and disk space.

The tech involved is actually pretty neat - the app stores the state of the zip decompressor (I just call those “checkpoints”) on disk every few seconds, which means it can resume even if you quit the app and restart it, or if your computer is shut down.

For formats like .7z and .rar, “streaming decompression” is impractical due to how those formats are designed. The best we could do is save checkpoints after every entry in the archive. And some games, like UE4 games, typically ship one gigantic .pak file and then a few small files. If downloading was paused in the middle of extracting the big .pak file, then that progress would be lost.

So we don’t do that. For those formats, like .7z and .rar, we just download the entire archive file to disk first. And then we decompress from the disk, to the disk. Which means you see the progress bar go from 0 to 100 twice. Once when it’s downloading, and the second time when it’s extracting.

So - as long as everything is working as expected - the only time you should see a progress bar restart from zero when you resume it, is if it was 1) a .7z/.rar file 2) the file was fully downloaded to disk already 3) it was in the middle of extracting it.

Even then, we do save checkpoints while extracting from disk to disk - to avoid re-extracting the same files over again on resume. So as an additional condition, it would have to be an upload from an UE4 game, or another game that has is mostly one huge file.

So I guess my question is: what were you trying to install? I’ll definitely want to try and see if I can reproduce the problem locally.

Like Reply

reddesertdisk4 years ago (1 edit)

Thank you for the response.

The game is Type Dreams by Hofmeier (https://hofmeier.itch.io/type-dreams)

I've got a few more questions if you don't mind.

Am I right to assume that :

1) Even when the progress bar restarts from zero when resumed, it doesn't necessarily mean that it's starting from the beginning (or re-downloads everything from the beginning) and merely resumes from where it had stopped downloading/extracting.

2) When it still displays 'KB/s - minutes' it means that it's still downloading, not extracting.

3) When I checked 'AppData\Roaming\itch\apps\type-dreams' there were files in that folder. Does this mean that these were the files that were downloaded, and will not need to be re-downloaded by the app again?

Like Reply

Amos4 years ago

Type Dreams is distributed as a .zip file, so the “good” strategy is used - the .zip is never stored on disk, and instead extracted on the fly.

In that case, you’d see all the files on disk as soon as the download starts. They’re just preallocated at the start of the single download+extract phase. (That should answer 3)

For 2, you’d only see a second progress bar if the format was something other than .zip, so not the case here.

For 1, well, if you had a progress bar restart from zero while downloading Type Dreams, something else went wrong, and I might be interested in looking at your app log: you can find it in “Preferences” => “Advanced” => “Open app log”, from that window you can type “Open folder” to locate the file. Or you might be able to diagnose just from reading the messages :)

Like Reply

reddesertdisk4 years ago

I just pasted the part where I saw 'Resuming @ 57.7%' change to 'Resuming @ 8.4%' later on

https://pastebin.com/wVC9AHwu

Is this enough or should I post the whole log? and if so where should it be posted?

Thanks again! :)

Like Reply

Amos4 years ago

Thanks for the log. Still, it’s kinda hard to tell what’s going on.. I did notice a bunch of network errors when the app was doing its self-update check, and when downloading locales…

Do you have any third-party antivirus running?

Like Reply

reddesertdisk4 years ago

Here's the rest of the Itch log, I hope it's not that daunting to sift through.

I did turn off all antivirus apps when I downloaded through the app, even Windows Defender and Firewall, so that they couldn't share bandwidth.

I do have something turned on called IObit's 'Driver Booster,' though I'm not quite sure if this is the one causing trouble.

If there's really no solution I just might revert back to my browser-IDM plan and hope the zip doesn't come out corrupted...

Thanks anyway for taking the time for answering my questions! :)

Like Reply