That was my biggest issue with this as well, and something I feel like was improved a good deal in the follow-up title Fisherclicking (https://filthyandfree.itch.io/fisherclicking).
