It doesn't really need to swap the whole texture page, it does get all the sprites from the same page. It only swaps for the goddesses which I put on a separate page because the sprites are very large and they wouldn't fit on the main page.
The reason it was doing a lot of swaps before v9.8 was because I was using the built in draw_rectangle function which swaps textures to a built in page that contains only a white pixel which it uses to draw the rectangles (I wasn't aware of this). Rectangles were used for progress bars on structures, so it had to swap a few times for every structure that had a progress bar. The way I fixed it was using a white pixel I had on the main page, and just making my own rectangle function.