this took me a while to figure out as well, but when you see the "angry" red speech bubble appear over your digimon, you have to tap them to avoid friendship going down!
So I've been worried and about this, too. Does tapping them too much when they don't want also make friendship go down? It's a bit hard to balance without any indication?