That's strange... Not seen any problems with my U64. I'll give it a thorough check later.
Probably related to the cartridge settings in the Ultimate 64 board. I switched the "Bus Operation Mode" from "Quiet" to "Dyn. & Writes" and it works stable now.
That's good to hear.