I'd say you're missing out on a lot. Think of it this way: Initiating combat doesn't necessarily mean that you're going to be a mean dragon. It more or less establishes who gets to call the shots afterwards.
Take one of the Eastern Dragons, for example. They view combat as an art form, and eagerly invite you to join them, because they think it's fun. Combat for dragons is often seen as less as a sign of aggression (territorial disputes notwithstanding) and more of a contest.