When you grab the ball, you take its momentum. It’s also you that has to go into the hoop rather than the ball.