Thanks for your concern! The way you lose is by a house/car not getting to their destination on time! Indeed when you stop time you can't lose, but this also makes you not progress in the game.