To the right of the bridge facing the temple, there's some blocks and little platforms you can scale up to get back up where you came from. I probably should have put an arrow to them, or added some on the left too.
Thank you for playing and giving feedback!