Ah cool, I wouldn't do a grim favor there because then something bad happened to the players and they had no choice or decision point that led to it. I would just have the crows swarm them and and start picking at them aggressively. That puts the ball in the players' court. What do they do? If they want to drive them off, that's an action roll. Maybe they goblins will scare them crows off, but maybe they'll panic.
You can still resolve that in a single roll, rather than an extended conflict, but they players are driving the action. They know what's at risk and they have the opportunity to try avoid it. Maybe things will spiral from there, but maybe not. Alternately, if you just want to convey 'this place is scary' you can do that in your description, without touching the mechanics.
Your instinct not to inflict conditons without a roll is spot-on. Goblins mark conditions from failed rolls, or if the fourth turn ends and they haven't made camp. Conditions are pretty bad, and they snowball sometimes. Keep throwing problems at the goblins and you'll see conditions stack up quickly enough.