General: Nice replication of the original paper. Shows great prompting effects. Runs interesting experiments that are well-defined. Nicely expands on the original paper.
Fazl : I like the notebook and it allows us to re-run the experiment.
It would be nice to have an intro at the beginning.
Alignment: Not too much.
AI Psychology: Shows interesting mathematical prompting modulations.
Novelty: These experiments seem like they would be done before since they’re about pre-prompting for multiple steps. The specific pre-prompting seems quite alright.
Generality: Not necessarily super general beyond the specific dataset but the principle of pre-prompt engineering is well represented as a general effector on the output.
Replicability: The report is a literal ipynb which is nice. We also expect to replicate it because it replicates from another paper and see good results.
Comments
General: Nice replication of the original paper. Shows great prompting effects. Runs interesting experiments that are well-defined. Nicely expands on the original paper.
Fazl : I like the notebook and it allows us to re-run the experiment.
It would be nice to have an intro at the beginning.
Alignment: Not too much.
AI Psychology: Shows interesting mathematical prompting modulations.
Novelty: These experiments seem like they would be done before since they’re about pre-prompting for multiple steps. The specific pre-prompting seems quite alright.
Generality: Not necessarily super general beyond the specific dataset but the principle of pre-prompt engineering is well represented as a general effector on the output. Replicability: The report is a literal ipynb which is nice. We also expect to replicate it because it replicates from another paper and see good results.