I noticed I didn't have the issue when I don't have a look target set, for now I don't need it for my project, but I'll probably figure out why in the future.