Not sure what I did, but the goldfish blocking the bullet hell hand/door suddenly disappeared after I "restarted" walking loops another 3-4 times. I think it has to do with the time needing to be 12:45ish or maybe 6pm, not sure.