Took me a bit to realize you could only have 6 walls - once I figured that out things went smoothly.
You can also mine rocks to build more walls.
oh! I need to go back and try that!