The random spawning of memories meant they often spawned right next to the exterior edge, and I had no time and no room to defend them. :(Cute capybara though.
