I think that you should drink a warmth potion as well. I drank two and with that I was able to travel
yeah it's a warmth potion + the clothes that let you move in the mountains.
