yeah thats what i am always doing but apparently if im doing any action with them i make them fear me, like it litterally says that if i succeed in the action i get -1 fear and then they have a disgusted look
It may be because of what you do to other NPCs just because you are nice to one NPC does not mean they forgive you for what you did to another.