I spent like 20 minutes trying to find the tiny island again, and gave up. :D Oh well, I ended up finding so many other things in the process, and it's a wonderful game all around.
You can find the tiny island! Go to the boat, and swim northeast, it's just around there.
