Yeah, if the Happiness is very low, there are no (or very few) homes and there is no food, the population will shrink. But it will take a lot of time to actually get 0 pop (and it is not really worth it :P).
