This is like how I navigated when I was first learning to drive and had no GPS. Want to get groceries, but I'm already at the mall? Welp, I guess I'm driving all the way home first, because I only know one route to and from the grocery store, and it starts and ends at my driveway.
It does feel very silly that my horse can ride from the city to the Elvish Capital by lunchtime if I leave after breakfast, but it takes until the following morning to get from the Elvish Capital to the beastfolk settlement.
Edit: Worse is that if I first ride home from the Capital and then, as a separate trip, ride from home to the beastfolk, that takes less time than traveling straight from the Capital to the beastfolk! You see, from the Capital to Aliron is one trip that takes two turns, minus one from Stables; and then from Aliron to the beastfolk is another trip that takes two turns, again minus one from Stables; and so (2-1) plus (2-1) is the same as 1+1, which equals 2. But! From the Capital to the beastfolk is one trip that takes four turns (because the travel time is calculated as though I'm going home on the way, i.e. 2+2), minus one from Stables; and so 4-1 equal 3 turns.