I think that computing RigidBody physics is really demanding. Maybe I should have lower physics FPS since the game is not a fast shooter but a casual puzzle. Well, that's why jams are great learning opportunities!
If its any help for your debugging, the part where you place the pieces and trying to rotate them was the part where it really started to struggle, as well as the rolling with click and drag
