After some debugging I found that firefox was the problem. And also I found out that maybe the version of babylonjs used might drop frames in different browsers.