You're right on it! You can only recover IF you're Bleeding or Drained. So, it's the other way around. It's less effective (or impossible) to recover during a delve unless you have been recently harmed. This creates an incentive to get hit as few times as possible.
I see it as, if you're getting hit multiple times, there will be lasting complications that can't be healed by a little bit of first aid.
Though, you're welcome to add in some extra recovery. Maybe each Beat only heals 1 HP if you're not Bleeding, etc. Or each Beat could be half as effective.
But, to be as hardcore as possible, you can't just keep resting in the dungeon to recover. You're being pushed to have to escape to be able to survive.
The. Dungeons. Will. End. You.
But really! Have fun. And good question.