The dex build with dumping points into endurance - that's one of the reasons to keep durability where it is. Endurance gives a nice bonus to health as soon as you add it and on every level above that, but the cost is low damage and the experience that you described. That tradeoff is by design, you choose if you want to make it harder early for a bonus later on.
The bonus stats still apply but are not shown on the leveling screen (just like bonuses from items, or abilities).
It's not possible to change the way durability works, but it's possible to remove it completely or change its value on every item. There are quite a number of people on our Discord that have fun with the editors and there's basic documentation in Soulash/data/docs/index.html in the game folder if you need help with modding.