Its a fun little game. I like that in some levels you do not need all the keys to go the ladder. Level 17 is a killer but a i managed to beat it on the 5th attempt. I've encountered a bug on the 18th level. The UI was not present and i was stuck and couldn't move. Look at the screenshot.
For me, I would say its the right level of difficulty.
One thing that you might want to explore is to make a level select screen - so that if you are stuck at a level, you can practice that level instead of starting from the beginning.
I haven't played too many memory-based games but hope this helps.