You first have to talk with all swamp village people and report to the Oracle about their behavior. Then defeat a Gatoroid (those bodybuilder scalies) and show the head to the Oracle to make the village people friendly. After that you can check the quest board.
Accept "the elusive husband" quest, talk to the quest giver and then you can interact with the wooden pole to get a boat.