Man, this is such a cool framing for a rhythm game and fits the theme very well! Unfortunately, I feel like the gameplay doesn't quite hold up yet. Unless my sense of rhythm is super off, I think the notes oftentimes just don't align with the music or even the beat. Also, a lot of rhythm games show the beat alongside the notes and I think that helps your players so much to put the notes into context. DJKL is an interesting choice, but I found it worked better than I expected; I only got confused a couple of times, while I had much more problems with a game using the more obvious DFJK layout.
I think there's also an interesting question of how to "balance" this. Since your damage is determined by the last couple of notes that you've played, this means that notes in faster sections have less impact on the damage, which is funny, as they are usually the hardest ones to hit. Then again, if you have a long section of silence, it means that the last few notes of the previous section matter so much. A reason I had so much trouble with the second song is that I often messed up the end of the first section and then had to watch as my hero just did a lot of bad hits in a row, without any way for me to do something about it. It's an interesting system as it is, but I feel like it doesn't reward good play enough, while it punished bad play in certain sections too harshly.
Anyway, I'd love to see this idea elaborated. I feel like there might be a lot RPG-inspired mechanics you could add to this to make it stand out from other rhythm games. Just a simple example, equip your heroes with different weapons and either go for a safe, standard choice or have one that has a lower base damage, but profits immensly from a filled combo-meter. Or adding a shield that allows you to miss some notes without losing your combo at the expense of slower attack speed. What I'm trying to say is, this feels like a super-cool starting point to a potentially awesome game.