I have been unable to find him
Im assuming you mean the worm guy. Well you can find him in Selthero's Den and instead of going down the rope, simple go to the right and you will find a small cave. Olive is super easy to defeat and he is one of those characters that sort of has no stat limits as long as you have the gold to pay for all his training. On top of that if you get his relationship high enough he will open up a shop.