What's important are indeed the hours. You get the amount of progress your slaves do, in hourly rates. Let's say your slave gets 20 progress towards fishing per hour and a fish needs like 8 each then if you pass the time by one hour, then your fishing slave caught 2 fishes and has progress for the 3rd catch. (these are random numbers i'm using here and the slaves rate gets modifiers with tools, classes and stat profiencies and some skills can boost effiency too.)
For other things, you need days. Quests are tracked by days, assigment quests too, and you can take new quests every 7 day. Most social abilities reset daily at midnight (they are needed to control the loyalty and obedience of your slaves, as you can only date one slave a day for a big boost in those stats and use social skills for the rest).
To be fair, i can't really explain it better, you need to experience it for yourself to understand it fully.