I guess avoiding the succubi doesn't affect the story. I actually don't even have any intention of looking for the brothel (or whatever they call the place where they work)
Avoiding the succubi is no problem, but in some story quests, you might have to visit the succubus brothel. Otherwise, you can completely ignore it.
The day the game forces me to go will be the day I find out where it is.
