Yes you can, but you are likely to have to fight a few times before the Merchant stops to let you buy something. I always just go to the Witch. Once her hut is open there are no obstacles to visiting her.
