Either that or like citizens trying to defend the city with torches or something. I'm imagining stuff where you can lose part of your swarm but not the whole thing instantly. And the resources in the city are more scarce so you have to use the farm area as like a preparing ground before you go in.