Thanks!
what i did was (i have perspective camera, so for orthographic the first part is a bit different):

Vector3 targetPosition = Input.mousePosition;         
targetPosition.z = 10; //replace 10 with world zero    (difference from camera position to player position)     
//set the target position based on the mouse input position
targetPosition = playerCam.ScreenToWorldPoint(targetPosition);          
Vector3 direction = targetPosition - transform.position;         
float distance = maxSpeed * Time.deltaTime;          
if (direction.magnitude <= distance)             
    transform.position = targetPosition;         
else             
    transform.Translate(direction.normalized * distance, Space.World);

its quite hacky, probably not bug free but it works!

if something needs explaination i can, but i think the names of all the things explain what happens quite good.