I understand that and you might be right. You make a good point but that doesn't justify what he did...or rather what he tried to do. He probably has trust issues and all which is why he might be reluctant to relationships but he doesn't have the right to ruin a relationship or be an abusive partner. He literally ruined someone else's life just for his own selfish deeds.
And about the sex part, Even though he acts like a gentleman and doesn't force it, he is still trying to lead you on and manipulate you. Though he surprisingly puts more effort into it than an average person would. But I guess when you think about it, it's really up to you if you make the choice to go with him or not.