The brown doors can only be opened from one side, they're sort of like general progress doors making the map easier to traverse. While the grey doors can be opened with keys.