I too found it frustrating at first, but after getting used to it and seeing just how heavily the dice rolls fudge in favor of us, the players, it's crazy. Though every now and then, against one particular boss, it gets pretty rough.
Make sure you do a scout and get a report of the boss before fighting. If it comes back that they're aggressive, put a higher die in defense. If they're defensive, you can generally get away with the higher die in attack. And personally, I prefer Hidden Voices from clash, once I know they're defensive/offensive, I know where to put dice.
Then yes, buffs galore. (granted, I avoid the elixers...I don't know why...almost like there's so many indicators about not trusting him that I feel like I shouldn't be taking them? Looking at you TheArcadean *peer*)