I'm sure I'm not the only one that sucks at the cooking game, but for me I think the biggest thing that makes it hard is the notes are not in line with the song tempo.
They are sort of similar, but there seems to be a bit of a tempo drift. So some parts of songs have notes on time at the right points, and most of the rest of the song is way off. This makes it really off-putting.
(Would love if the difficulty was toned down a bit across the board, but if not, this would go a long way to making it better.)