Aight, so take this with a grain of salt, since I dunno if you're playing with active combat on, which in my opinion is way harder than passive, which is what I play on since I don't have a mouse and it makes things simpler.
Weaponry in this game scales hard with whatever stats affect them, but they are not made equal. Notably, the ranking (D, C, B, A, or S) determines how much damage you get from a weapon if you have a lot of points in the corresponding stat. Furthermore, not all stats are made equal. Bust, as a prime example, gives you tons of AP to play with, and lets you hit enemies with the wrath of god after using massive buffs and potentially even stealing growth from your enemies... but if you get hit, which is likely since bust doesn't make you fast, you get flattened at an impressive pace. On the flipside, legs builds let you go first and take twice as many turns as your opponent, along with dodging attacks, but if for whatever reason your opponent can debuff you, withstand your attacks, or is just plain too big, you get squashed. (Thankfully the Bard's Beak weapon from the blacksmith deals unholy amounts of damage and has an attack that seems to ignore armor, and starts at B-rank scaling I think? It's a lil insane.) With all that said, if you're finding enemies to be inordinately difficult, try switching up your loadout. I'm currently at 50 in every stat except for legs, which I cranked to 300, but I can take down the boss rats in the sewer up to floor 25, and those things are huge.
Also, I highly recommend unlocking as many things in the ability trees as you can, as soon as you can. The bust tree in particular has some crazy boosts, and I've heard that the Step skill from the size tree can win fights like nothin else if you're running a size build.