It can definitely take awhile to get used to, but I think it works well once you know it. I'm glad you got the hang of it.
I do plan on adding some more QOL features to make understanding and mastering the mechanic a bit easier, or a more in depth tutorial to gradually teach the mechanics.
I did try to at least have one good shortcut within each level, so going back to get more time for the later level could yield players finding them.