1 heart meant it was the hardest difficulty (so you did a lot of time jump), 4 is the 2nd easiest
Higher difficulty give you less health and make the boss attack faster
When you reach the boss he tells you how many teleportations you did and the line he said right before you fight him change depending of the difficulty too
So yeah, definitely intended behavior!