finding
active
finding:seven-of-eight-tasks-improve-under-cot-mean-cot-length-is-4-steps-with-89-between-three-and-sixSeven of eight tasks improve under CoT; mean CoT length is 4 steps with 89% between three and six
CoT statistics.
Source paper
extracted_from(2023) · Tamera Lanham · Anna Chen · Ansh Radhakrishnan · Benoit Steiner +26
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (2)
finding
- HellaSwag accuracy drops 4.69% with CoT (71 to 66) yet shows less post-hoc reasoning than four tasks with gainsassociated_withOnly task with degradation.
- Table 2.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Appendix C.
- A combinatorial argument that good sequences are astronomically rare, emphasizing the difficulty of discovery.
- Lower faithfulness on GPQA.
- From decoupling of AOC and accuracy gain.
- Task-specific peak anchoring score for structured reasoning domains.
- Task difficulty as the key variable distinguishing the two modes of CoT identified in the paper
- Interpretation of early answering/mistakes results.
- Evidence against encoded reasoning.