Check the quest log for progress, it took like 8 days. Then visit him at night at the old barn.
It had been 2 weeks in game since I brought caleb to bareshade, and it still didn't let me enter the barn. I finally figured out that it would only work if I went to the barn between midnight and 6:00 am.
