same thing for me too :( i think it's a bug
i've tried sleeping at the inn and sleeping in my shrine so the day can arrive but as soon as i step foot in madrana again the figure tells me it's gonna happen in 3 days and the counter resets