It's not possible to skip or decline the quest. If you entered the forest but picked to go "later" to the festival, then going to that exact same entrance again will give you the option again to start the festival. It does so until you accept it. You should still have the quest in your quest log as well.
The position of characters in your formation is never important for anything. It only matters if you have obtained a character, not if he's in the active battle formation.