Yeah it's based on sanity, losing to them when they have high libido makes the scene happen, when your sanity is low enough you get a sanity ending. There are a couple sanity endings and I think I might know them all, if I missed one or another please respond with it!
The Werewolf alpha, Werewolf L, The Troll in the bridge, Raptors, Roushk before he's purified, any slimes, I've heard Viccoria might have one or she just lowers sanity, the tar beings in Viccorias caves and I think that might be it. If you find another one please tell me!