First you have to train it until you have mastered "ride" then you have to take it.
You should see it in your equipment window.
And don't forget to take it off when you are done.
I understand you can't enter the underground on a horse.
Just so you know, the map will still reflect the walking time but you will move faster as seen by the game clock.