You can say no, but ultimately, even if you get the other two on your side, which neither of them would want to say no to the first decision of a united army, you still somehow don't get the majority. This means you have to go with the duke's demands no matter what if you want the united army.