Really awesome concept and great execution! The core game loop is really fun, although it might be a bit hard punishing some time. I felt like the only way to win was to get lucky with the building, and once you're set up you become to big to fail. But I had loads of fun, managed to beat it around my tenth try!
The footsteps sound are a bit unnerving when you get more than a dozen, so maybe always play the same sound without regard to how many followers the player has, I feel like the visual indication is enough! :)
Great game, really happy I played it!