Pretty sure he doesn't say that. And losing progresses the game too.
NO, he said:
"Those guys at the big table over there. They are the guys you are looking for. their boss is the one you'd want to squeeze information out from, the bald guy."
Nothing about winning, and if you lot to them you should be getting an item called Bandits' Hideout Location already, which will help you progress the game.