You need higher Innocence when talking to him, I assume I set it so the quest (A Switch) advances after talking with him with low innocence, making the quest not possible to complete, if so I will make it so increasing innocence again through the playthrough makes it possible to finish it
