This is very hard. After playing about 10 levels my brain still didn't develop connection between keys and colors. Maybe constant action binding to key (J/I is always for moving left/right, E/F for switching doors) would work better. Also I would like to see tips for color/key always on screen.
This is very good idea with very nice visuals!