You need to equip a torch or acquire the glowing amulet from the Lizard Tribe questline to be able to go to caves.
