I think automatically after a while the priests will just forcefully exorcise the ghost from you. I don't think you can remove him forcefully. I could be wrong, unless he gets enough salt thrown at him.
