Don't worry, make the game you enjoy making and the market will follow, as long as you keep the game in line with your core ideas you're golden. As for your audience, it's me, I'm your audience, and I love good controlling rhythm games and I know there are lots of others. I believe you can make something you and in turn lots of people enjoy. :)