If you're over a certain size you'll either need to wear a wizard hat or use growth spells on him in order for him to trust you
