You can move your existing buildings around, you need to defend & get more gold making buildings to get new buildings. The upgrades are having an effect, it's sometimes just stat changes
I replayed it and now a understand it better. Maybe I was misled by false expectations. It's a very nice game and I love the pixel art style.
