I saw a crash in 1.2.4 around fighting masklings, still trying to chase it down.
Migrants idle when they reach the settlement's influence range (the distance at which you can construct buildings). That influence range is based on population, and can get pretty big. Those idle migrants will stop idling as soon as they're assigned a task.