Yes yuo can I ended up with 3 maid dresses due to dismissing it. So you can take it's weapon and it will get a new one once you have him stop following and reload the overworld. I just wish he could get stronger than he is. Guess here's hoping a new update allows it