Winner of the third prize! Congratulations!!
General: Really nice experiment that represents and experiments the concept well. Missing some sort of graph, though the results were there as a table (sort of a graph). The data goes deep as well and invites a lot of further investigation!
Alignment: Represents a basic language model error very in-depth.
AI Psychology: Replicates an interesting symbolic re-definition principle. Has an interesting method for evaluating
Novelty: Not seen before in this format. Syllogisms have been seen before and the symbolic redefinition problem has been seen before. But describing these in a new combination, i.e. with nonsense vs. original syllogisms.
Generality: Many datasets tested with different question formats and prompts. Very nice. Can imagine the further generality from the results. The syllogisms were mostly in one form, though. Replicability: Code, data, and results laid out in very neat form.