Hey! I have the same idea but ended up doind something a little bit different, I like the concept, I just wish that the book with the color is always on screen.
Eyyy, great minds think alike! I debated on doing that, but I felt that having to memorize the colors and having the villagers text not be super simple gave the game a bit more complexity, which it's lacking a bit.