Interesting idea, however I feel that a quick warning before the change is in order (just a few milliseconds either a sound cue or a slight flash or background color change). This way it gets quickly to frustrating as the player doesn't have a chance to "get it right"....