Yeah, sometimes you've gotta just kinda say 'no' to feedback if it goes against your vision. Not to say feedback should ever be completely ignored, but with niche genres you'll definitely have to go directly against popular opinions sometimes. Make a game you want to play, that you can be proud of, and you're 100% gmi. You definitely know your target audience, and that audience is a bit on the small/niche side - and that's okay. You don't need to appeal to everyone, just that small group. You got this, so don't give up!!! It's REALLY fun.