Are you seeing nearby pigs that they aren't fetching? Gathering pigs is one of the tasks that citizens won't do while under threat., and they only look a little bit outside the settlement's influence range.
Maybe the pigs I was seeing were out of range because they are gathering again now :)
