XD That was neat!
I think the ghost-chasing part should be muuuch more shorter. Running in circles for minutes is not fun! Maybe each ghost will be able to take just one “hit”, and create a visual effect or something to be more intuitive.
There is little challenge in walking in circles. It was more a puzzle (remember about the medallion, understand what is needed to defeat ghosts) than a challenge, as ghosts are easy to dodge as they tend to walk into each other and form a small blob of death.
And as it still has a puzzle element, I would not add a skip button. But yes, something must be done with this part of the game as I do see how it can disengage people. Maybe you can make ghosts more interesting instead.