This is a cool concept! I love the mystery and the birb. I think what the issue with difficulty comes down to is that you expect very precise movements with a very imprecise controller. This feels like a cursed issue where you want some imprecise controls to make for interesting gameplay, but then the difficulty of designing interesting levels with that restriction is huge.
I think a helpful change as simple as having the rotation snap to say, 10 degree increments, or the movement being automatic so all you have to focus on is the rotation, or maybe even reorganizing the controller. I felt like I hit stop or play (which have almost the same function) every time I was trying to jump back to turn the other direction. Something to make the controls just slightly more precise would really make this game 10/10 for me. Right now, I'd say 8/10 for sure!