Yeah, I was trying to debug it but turns out the way I was using the raycasts made it not register far away objects.
