Maybe no restarts but some fun dialogue/event (maybe appear in another place fast after collision: ghost acting as a portal for misplaced items) or just something to spook him off ("Here, ghost, I hit thee in the nose!")).
That is a good idea, it will make the ghost even cuter XD