So long as they’re colours from one palette, yes you can use less than the 4.
Got it! Thank you for the response :)