Just in case someone has this problem, it has to do with the render camera.
It also breaks some other things like onMouseDown() not working or screen to world point/viewport whatever bring back unusable coordinates.
Using the simple method, without lighting support(1 main camera + pixel perfect camera attached to it), gets rid of all these problems.