Oh, I wouldn't worry too much. If you like what I've been doing so far, you'll be loving what I have planned for the future, so no need to fret! :V
My only problem is that right now is that right now the situation seems to be "it can only get worse". I was hoping a certain person would be able to achieve their dream. If the game plans to be about 1 man getting a harem of girls, I dunno about continuing this next chapter. Not sure where this game is going.
I'd say give it time or even let it sit a bit and come back a bit later if you're worried. Like yeah, at this point in the story, it's a rather low point for sure. But I wanted to make a stories with highs and lows after all :V Will it stay at a low point for the cast? Nah, of course not. Not my style :V