Wearing good armour will help with a physical altercation (and/or helping you survive long enough to win via tease). Try to focus a little on health as you level up. Finally remember - you're an herbalist! Be sure before you set out that you have a healthy supply of strong healing potions, and then ideally some others. You'll want to get the healer down first.