I analysed my difficulties more and noticed that the negative happiness spiral is always the following:
1) Happiness is going doing.
2) Raising happiness would require buildings which need a high number of people (parks and restaurants).
3) Because of the happiness level, I can't get enough people to join my colony to raise the happiness.
4) Happiness keeps going down which is the end OR I ran out of food because farms also require people.
Improvement suggestion: Adding a building option to improve happiness which doesn't require people at all.