I ended up fixing the poor frame rate by moving the build off of WebGL/HTML5. For some reason this build had quite unplayable frame rate and the only way to fix it in this time was to move the build to PC/Linux/Mac. I wish I would have been able to make it playable in browser so it would be more accessible, but I would rather allow people to enjoy it rather than be frustrated with a poor build.
Maybe in my next game I can work more on performance tweaking so this doesn't happen. Regardless I hope you enjoy!