Did you encourage him to go to bandit's camp in Bandit attack event (Logan bottom route)? If that so, you can't choose him. You can only choose to follow him in normal route (by discourage him).
