I did do that though, and it just kept switching between North and South, even as a test I pressed North once and then South once and I still didn't return and it just said that South was now depart and North was return
it changes between Return/Depart depending on the last direction you went (If you went North, now it becomes Depart and vice versa) So if you want to get back to whitefang you must continue pressing "South" regardless of what it say x3
Also, depending on the text that displays on top you can know how far you are from reaching Whitefang. If you went North once, you must go South twice to get back