To be frank man, I didn't notice on my first attempt that those time orbs were dropping at all. I didn't know the enemies dropped anything until I saw your reply. Now that I know that I gave it another go, and I didn't really have that issue anymore. Maybe the root issue is that the presence of those time orbs weren't really communicated to the player. That's an issue that could be solved in lots of fun ways!
For example, pickups in other games use things like motion, sounds, and other aspects to tell the player where they are and that they should pick them up. They spin around, bounce or hover up and down in some cases. Sometimes they'll emit particles or sounds to grab the player's attention. Another important thing is feedback. Making sounds and effects when pickups are collected can inform the player that it was collected and what the pickup does.
Of course alot of those solutions are polishey things that often don't have time to get implemented in jam games, and I understand that. You did quite a lot with your limited time, and that's something to be respected.
Thanks for making a great game! (Also sorry for the text wall, I'm pretty sure you didn't ask for this ._, )