The thing with the fortress is that basically it should have worked but you picked one of a small number of places on the map you could be up against the edge and can't wall off the other sides. The walls are ideally like a plus sign around a unit. The up, down, left, and right will be shielded by a building at any distance but the corners of the map are not safe ever.