It seems you have some heavy task before you schedule jobs in your system. eg. allocate a large array. To figure out which piece of code takes time, you could use ProfilerMarker
https://docs.unity3d.com/ScriptReference/Unity.Profiling.ProfilerMarker.html