Make sure you use the 3D collision checking script (so you check the bullet's Z position as well) instead of just a collision event.
I don't think the bullet displacement will be noticeable in-game. Right now the bullets are slow and their projectile model doesn't rotate, so perspective plays some tricks with you. And many types of guns (machine guns, shotguns, grenade launchers, homing projectiles) won't shoot in a straight line anyway.