Firstly, are you stealing from guarded faction chests or releasing faction prisoners while observed? That'll make even a friendly faction's members attack you, and they really will mean it! If you value your relations with a given faction, don't steal from them or free their prisoners while they're watching! Doing so in secret may also give you a penalty, but it won't be as bad as if you were caught in the act and also had to disable a few guards!
Otherwise, members of neutral or even friendly factions might occasionally have a dominant mood and 'playfully' attack you, chasing you down to tie you up and tease you a little. You have the option to retaliate, but you will face a rep penalty if you capture them; they were just playing, after all. This is especially true if you fight back using lethal damage; no one likes it if you actually murder their girls instead of just tying them up! If you avoid a dominant attacker for a little while they'll give up, and disabling them with your own bindings without capturing them is totally fair and incurs no penalty. That said, the rep penalty isn't very high, so you might consider capturing them anyways for the ransom value.
Note that the flipside of dominant moods is submissive moods. Sometimes, a girl will get that heart over her head and say something like "oh no, I'm out of mana and totally helpless, I sure hope that nobody takes advantage and ties me up, tee hee~". At this point, if you have the bratty subs mode turned on, they'll 'playfully' run away from you, hoping that you'll chase them. Either way, these girls are literally asking for it, and you'll face no penalty for attacking them with tease/bind damage, and if you render them helpless in the process, they'll even drop their inventory as loot! The game knows that they're fair targets, and won't even prompt you before you attack them with suitable weapons, unlike with non-hostile NPCs in any other circumstance; even their fellow faction members won't object... so long as you don't take matters further than bondage and teasing!