It's kinda expected tbh. You have to go into making games with the expectation that it'll probably not get much attention. So when it does, you're pleasantly surprised. And if it doesn't, well you were prepared for that outcome so the disappointment is more manageable.
Just never give up. My first game took almost 2 years to even get to 280 downloads. My newest one got to that amount in just over a month.
It'll get easier the more you try.