I got to the caterpillar and couldn't for the life of me get the last two pixels filled in (with this game, I don't know whether that was a bug or a feature!).
The game reminds me of the ethos of Bennet Foddy's Getting Over It. As Foddy said of his game: "This isn't nice." So I can't in good conscience say that this is "fun", per se, but it certainly is genius on every other level. Sometimes the decisions you make on colour picking depends on what colours border the spot, which made for some amount of analysis, which was nice.
Very cool idea and implementation. Well done!