If you've got the sand armour and you can enter the desert, then you don't need to go back to the valley or check on the blacksmith.
Just head on into the desert and continue the story. You're looking for something in the north-east...
If you can't get into the desert because the sand wall is turning you back... then you need to go and talk to the Wizard at the Inn so he can activate the sand armour for you!
Hope that helps.