There's no built-in button to delete save files (I suck at UI design) but the savefile directory is C:\Users\[your username]\AppData\Local\TH_HollowHearts, you can delete the files there. (thhk_save0 is save slot 1, thhk_save1 is save slot 2 and so on)