you can't open the blue door, the purple door is open in the night and you need a amethyst fragment (the fragment is in the pit), the red door is open in the morning or day and you need the angel's feather (Haru gives it to you) and a ruby's fragment (is in the cave next to the read door) and the green door is open in morning and day and needs a emerald fragment (Mary gives it to you)
or (in regards to the green door) if you complete the shopkeepr's story first, or Haru's story, and go to the Goddess fountain ,you'll get the ability to cut down the vines, for instance the ones blocking the cave to the south of your house. That particular cave gets you an emerald crystal too without needing Mary
