I love this idea, it really scratches that exploration and puzzle solving itch. One issue I faced was that the scoring was a bit finicky and overly punishing, but I think that could be easily fixed with some balancing!
This is beyond the scope of the jam, but I would love to see a full game with multiple levels with different environments. I think that could add some interesting variety in navigation due to varying terrain, obstacle, and landmarks! It could also allow for some nice environmental storytelling!
Well done!