Totally, this definitely seems to be hardware dependent, and possibly browser dependent too. Thanks for your feedback.
I've tested this on a few different devices, safari browser and some other devices seem to struggle. I'm guessing it's because I'm leaning heavy into canvas filtering which I think may be the bottleneck here. I'm using the canvas filtering to color/recolor a lot of things so could be fixed if I took the time to create the new sprites instead of recolor them the way that I'm doing.