Nice project. I also wanted to do something like that for a long time but didn't at the end : you may have given me the will to try it myself.
The biggest problem I would have will be the creation of sprite animation without using 2d bones. I still have to mess around with stable diffusion and find a good workflow. Control Net might help but I don't know.
Anyway, thanks again for sharing this.
For the main player I wrote also this: https://civitai.com/articles/2609/some-tips-for-making-a-2d-game-with-stable-diffusion-graphics-in-unity
Net control works well with bones but the details change a lot for frame to frame. Best will be to make a specific lora for your player or enemy and then using it..
I already read your article on Civitai : it was a nice one.
Concerning the animation, maybe I'll try to make it on the same picture like other suggested some time ago on Reddit to keep the consistency. And yes, having a LORA is a huge benefit too. But I think the IP Adapter Control net would be able to give similar result. I had great results with this whithout using a specific character LORA.
