It installs the new game without removing the old version, so you have to delete the old folder manually, but it does *not* get stuck if you *close* the Itch client out and ten delete it.
It happens, though the top comment seems to be worried about a lock on files open in an application, which is a pretty common thing to keep you from breaking active things on your PC even in Linux...
Though a root user can easily override that while a Windows Admin can not.
Even Firefox has a lock in the directory that makes it impossible to delete, even as an admin, while the browser is running.