The size of the site does matter right now, since Steam is pretty dominant on the market and there isn't much reason for people to look elsewhere unless they're really interested in the indie game scene themselves.
I will say that a large issue is marketing as well. Most indie developers hope that getting on a front page or having a banner ad someplace will do wonders, and while it may help, there's a lot more that you can do. Look at the Big List of Youtubers, see who would be interested in your game, watch a video or two of theirs to be sure, and send them an email. It's exhausting (at least for me; I still need to finish going through the list), but contacting people for reviews helps out.
Twitter helps too; if you're an indie developer and not on there, seriously consider getting an account just because it's so simple to just throw hashtags onto posts with screenshots and store pages to boost your reach. I managed to get a couple very small channels to do a review on Youtube because I asked publicly on Twitter (and they were even pretty positive, like this one!). Even if the reviewer has no subscribers, don't underestimate the value of someone being able to type in the name of your game and be able to see someone else play and examine it.
Also, pitch your game at the drop of a hat. If a stranger comments on your shirt because they recognize and like the band on it, mention how you like whatever style of music it is, but that you haven't had much time to listen to them since you've been working on your game. Learn how to segway discussions with potential players toward your game and what makes it so interesting. Always have pen and paper on hand to give your game's url to people. It may seem insignificant, but I've made more than one sale and several downloads this way.
I'm sure there's a couple more factors as to why it's hard to find success on here, but I think the site size (and relative obscurity in comparison to places like Steam and GOG) and the average indie dev's marketing strategy to be the main issues.