Thanks for giving it go.
The ladders are climbable while Holding Left Shift.
I had some issues with the climbing mechanic, since when you reach the roof you still be able "climb" on them.
So it wasn't a straight forward up/down ladder.
Definitely something I want to continue improving on, as it would be awesome to nail it down for use in other games.