Did you get the Fejora Crystal? After picking up the crystal and beating the fairy, you should be able to go back to the village.
There was no crystal I could pick up. Was it that sparkling thing next to the broken crystal? I couldn't interact with it. I used a different save file the next time I saved, so I can easily go back therer and see what happens. The only things I could interact with was the door, the fairy, and the entrance to the village, where I was only told to complete my objective. I'll load the save again and check later, but I don't think I can do anything.
Also, I originally wanted to keep this exploit discreet for my own benefit, but with the charms I got in the Xeoclat Hills, I no longer maintain status effects outside of battle. All I have to do is equip them as soon as the battle is over, then unequip and equip on my next party member that has it, and it immediately takes away the effect.
Edit: I loaded the save and it seems to work now, however, I think I'll use the save that I chose to wait for Uni because I like tanking better than a large amount of attack power.