I don't want to rewrite my shader library.

Absolutely! Using existing free / open source assets and listing them in your entry's description is encouraged. 

I use Unity for development. Let’s say I own a couple of asset packs (paid) from the unity store with models for creating background buildings, can I use this, or are only free assets allowed?

I think that's fine. Assuming you're unable to share them in a public repository, I'd suggest mentioning any broken dependencies or missing assets on your game page / repository. Bonus points if you share where to get them and/or provide placeholder assets :)