Don't read too much into a single experience of your game.
I think what irritated me about the clock was, that night and day of course do not correspond to 0-12 and 12-0. But there is daylight twice with some hours and the irregular sleep times disorient me.
Since you added night effects, you might want to consider using the sun as an indicator. Sun and moon wandering across the sreen. Or as a background of the clock or whatever. Or a more granular night and day, with dusk and dawn in between. Speaking about the lack of light on the map in night.
It is mostly the sleep times I think. If a day is scheduled regularly, I do not need to think about what time it is after waking up and have some inkling what time it is depending on how many things I did. What could do the trick is tell me how long I slept or rather what time I woke up. There is a sleep dialoge anyway. It could end with something like, you sleept till 6:00.