There are terrain tile effects. Like if you place a card next to a Desert, it would trigger a discard from the hand. And placing a building next to a Swamp gives it -3. If a building lost 3 points when another was placed, the only combination I can think of is Factory (-2 to nearby buildings), being placed nearby a Barn, which loses 1 anyway each turn. I'll work on making these changes easier to follow.
Thanks for playing and the feedback!