Ah! So you can only fish when you are in proximity of a fishing spot. I suspect what happened is your boat kept drifting forward while you were fishing and left the spot.
Ooh makes sense, I think stopping the boat automatically for the player could be Quality Of Life improvement, but making the whole game more realistic and challenging is also a possible way where to aim your game! :)