when I rejected I had to pay 1000 coins later to make the spirits happy again
You don't have to do that, the only downside is that you lose some happiness over time.
