Unless the silence is intended, always add them. It doesn’t take much, unless you want very weird sounds.
