tl;dr I didn't have skinning coded properly for Adreno 650 (GPU).
More details: According to GLSL ES specification, for vertex shaders one should be able to index any array with a variable. And so it was implemented. An array of matrices (bones, mat4). Adreno 650 does not support that and one of the solutions is to have an array of vectors (vec4) and when a matrix is required, to construct a matrix out of 4 vectors.
It may affect the performance but still Adreno 650 used in Quest 2 is much more powerful than Quest 1's Adreno 540.