After changing their orientation in the first map, it looked like I could walk right in front of them without anything happening (I don't remember all the details / procedure I did sorry)
Ah okay, that's allg. That's probably the fact that I added in a small grace period and made it too long. I wanted to make it so you weren't instantly punished for hitting their vision cone, but I set the default too high most likely.