If you give to the madam, she will like it and give you possibility to have romance with her, if you give it to the gardener, he will be able to get in Madam favour, but he will give you a gift that you can trade with the doctor and he will be cool with you in the future.