It still would not let me reach them for some reason
You need to place the ladder and then use the candle in your inventory while standing in front of it. The candle you're holding also needs to be lit. You can light it on one of the other torches.