It's not like an actual horror game. There are no monsters or creepy jumpscares / disturbing pictures. Still, it is a tense game and you might be taken by surprise (e.g. if the homeowner suddenly walks up to you). As you described it: someone is looking for you.
You can see in the gameplay videos in the comments below that other players were scared or nervous at first, but the more they played and learned, the less anxious they got. Others weren't afraid at all.