I've since found out to press "f" to "flame torch". This is listed in the game commands, but not the manual! :(
That said, I am a bit lost. The dungeons, in first person, can be very disorienting, and there doesn't seem to be any boss or final loot to give a sense of closure? So I never know whether I've cleared them or I just missed a path somewhere (and they all look the same, so navigation is a bit hellish haha).