I did get the boundary problem in one of my first playthroughs, it can help to keep the pets away from the edges I guess for now.
I played it again just now, was able to survive past the 3rd wave somehow (for several waves) and I didn't stop getting money after the 3rd wave.
If you saw the tree disappear, that might be a different glitch, it alternates between positive infinite money and negative infinite money, and that would affect you on all waves or the whole game once it starts to happen. You may want to try to avoid that glitch by not fertilizing the tree past a certain point (about 164 or higher rate, and the tree looks super huge).
You can also use the glitch for infinite money because there's moments you can't buy when it's negative, but there's moments when you can buy whatever you want because it's infinite positive. However, I think the glitch is unpredictable and you may not be able to buy stuff when you need it, so I see it as more of a hindrance than a "cheat" that gives any advantage. So I think it's better to just avoid it. I have some screenshot somewhere on how big the tree looks and how the rate number looks when you should stop fertilizing the tree to avoid the issue.
However - it's possible you got a different problem though if it was specifically after wave 3, but I didn't get that problem yet for now.
Viewing post in Petville (Brackeys Game jam 2024) jam comments
That is so bizarre, I never got that but I guess it might be some other problem. Probably there's not much to do but restart. In case I play it again I might keep an eye if I get that because if money just stops it might really not be possible to get around that one.
Only other thing I can think of is, and I just thought of it now actually, is that if usually playing the web build, maybe the Desktop build doesn't have the issue (but that's only if you can play that build). I did not get the issue myself and only played the web version, but it might just be a matter of time and I might get the problem as well.