Possible bug: Surveyor is supposed to add +2 to your Perception during Scouting/encounter rolls. In both the Bandit and Ruins encounters, when it is making Perception checks it is still based on the Base Perception value. Harpy trap encounter works fine - I think that's with Agility though.
Viewing post in Known Issues (and bug reports)
Hi - the Perception check with the Brigand is actually a regular perception check; it only occurs if you've already failed the scouting check (invisibly) which the Surveyor perk does help you pass. To pass it, you have to have an effective 5 perception with Surveyor and any scouting that you've done (each Scout action increases effective perception by 3).