There is no random loot for dueling. The other NPC captains that you can either insult, join for missions, are the same group of characters that you can select when you start a new game. If you don't select them to play yourself, they are still out in the Caribbean, taking "missions" themselves, and show up in various ports during their travels, giving opportunity to either team up, or as you discovered, simply dispatch in a duel. If you read the short bio for the characters, some are famous and notable, and others are not. Their starting gold, weapons and ships typically match their "success" level and is noted in their character descriptions. The loot you saw when you killed them, is the same loot you would have started with as that particular character, so it is not a random prize for the duel itself. Some of them were very famous historical figures, so they would have been wealthy from the start during this time period. For a few examples from the ERAS bios:
Henry Avery took the greatest single prize in history & he was never caught. Dubbed as the "Arch Pirate" & "King of Pirates"...
Gold: 600,000 Pieces of Eight
Equipment: Excellent equipment and very valuable items
Ship: Captured Mughul of India's treasure ship 74gun man-o-war "Ganj-i-sawai"
Level: 14
Michiel de Ruyter is one of the greatest commanders of history...
Gold: 80,000 Pieces of Eight
Equipment: Good blade and valuable items
Ship: 92gun Dutch 1st Rate Man-o-War/Flagship of Dutch Confederate Fleet "De Zeven Provincien"
Level: 16
Blas Miguel...is famous as being the lesser of the two Corsican brothers.
Gold: 3,000 Pieces of Eight
Equipment: Basic equipment only
Ship: 16gun Barque-Longue "Un-named"
Level: 1