This is what I did while testing it out: 1.) Do not interact with the baby first and then interact with the pepper.
2.) Interact with the baby very fast and then interact with any object.
The thing though is it is super random which is weird. But still, was able to play longer after 2-3 tries :D Still can't get over with the manager voice hahahaha