What I do is delete the APK file and download the new one
Never delete the installed game.
Although Deleting the old .apk(from your downloads) is fine
