Glad to here it! I think different enemies would be great, maybe moving at different speeds or patterns. And I think having the castle start as a hut or having any kind of upgrades/progression would instantly add a lot.
I did slightly randomize the speed, but it's not too visible here.
