I like to think of the point system as "his amount of trust" in the player. Since when you get a very high level of APs he acts like you are sure to love him as he loves you. On the other hand, at negative APs, he realises that he'll need to do... "more" to keep you close to him.