is there an option to not adopt then
You will be able to choose if you want to adopt or not!
Your spouse will present you with those options a while into being married. You'll only be presented with those options once. Your spouse will have an opinion on it.
If you agree with them (adopt when they want to adopt, or don't adopt if they don't want to), your unique marriage relationship with them will increase a bit.
If you go against them (adopt when they don't want to adopt, or don't adopt if they want to adopt), it will affect your marriage relationship negatively.
There are also quite a few spouses who don't care what you pick, which means that you can choose whatever you want without risking them getting upset with you - or like you more, for that matter.
No matter if you go with or against your spouse, it will only have a small impact on the marriage relationship, though (for most characters, at least. It will be obvious if the impact is bigger, because your spouse would get very upset if you go against them). It's generally pretty easy to make up for a potential negative impact caused by your choices with adoption. This is because you can increase the marriage relationship in the spouse's three unique married life events. But it's also possible to decrease the marriage relationship in those events, so it's important to listen to your spouse and understand them, if you want to maintain a good relationship even when married.
TLDR; It's ultimately up to you to decide to adopt or not. But since every character is unique, your spouse will have an opinion on the topic and it might affect your relationship with them in some way.
I hope that clears things up a bit!