When you're setting rb.velocity in FixedUpdate that will overwrite anything that happens in Update. FixedUpdate is called before every Physics update.
Thank you for the info, any way to surpass that?