Didn't they usually bring the ship to a stop by backing sail(s) and/or turning into the wind and then dropping anchor?
>All in all, it's the kind of situation that would make me want to create some sort of artificial braking effect when the sail area is under a certain threshold
I would think it's possible to condition the drag not only by sail area, but also by checking if the ship accelerates or decelerates.
P.S. Here are two articles concerning period ship maneuverability I found recently - maybe they can be of help.
https://www.cnrs-scrn.org/northern_mariner/vol13/tnm_13_4_29-39.pdf
https://www.cnrs-scrn.org/northern_mariner/vol14/tnm_14_3_57-68.pdf