it should load up the appdata folder, where you'll find the RenPy one with all the saves to delete. You might need to delete the ones saved locally alongside the actual game files too, since I believe they copy over.
afaik they do. And sometimes the folder in the renpy folder is not named like the game.
... \AppData\Roaming\RenPy has a folder for each game
and as a backup the local copy within the installed game folder.
... \game\saves
Though the saves should be deleteable from within the game, but not necessarily the overal status of the game and maybe not the autosaves. It saves stuff like which page was visited already for skipping and other things, like achievements the developer put into the game. That would be the "persistent" file in the save folders.