based on the status screen i gave a guess to it its probably this
also heres a tip for the hobos in particular face back to them they will run at you when back is turned then spin around and strike them for goblins its really just a case of time the attack for when they are in front of you and utilize door ways to force them to only go in front of you as for bandits they will just run at and assault you and good early game stats to focus into is SCU at least to 10 or so to get improved throw rock and con gives you a better time due to the expanded sta pool but for early game only really SCU and traits matter