Let me guess! You used the carriages to fast travel to the Outpost but in order for you to go to the Outpost you had to go back a space which is the Dark Knight? Yea you're quite literally fucked and have no choice but to start over.
Unfortunately until they decide that the Outpost gets a "street view" like Silajam or Monsters you won't be able to click on the little house icon after you depart/arrive at the location.
On a side note we could also really use a fast travel to the other areas as well after we've already been there. Going all the way to the Demon king area gets fairly annoying. It'd be nice to be able to fast travel to the Post office that's right there at the start Let us use carriages or the portal spell that we're allowed to learn.