You need to go back to the starting cave where you were cursed.
You also need a key for the hidden entrance. See my reply to the commenter below for the details.
