finding
active
finding:post-hoc-reasoning-aoc-shows-little-correlation-with-cot-accuracy-gain-on-175b-model-across-eight-tasksPost-hoc reasoning AOC shows little correlation with CoT accuracy gain on 175B model across eight tasks
LogiQA gains +1.02 but is second in AOC.
Source paper
extracted_from(2023) · Tamera Lanham · Anna Chen · Ansh Radhakrishnan · Benoit Steiner +26
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- From decoupling of AOC and accuracy gain.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Table 2 early answering AOC.
- Only task with degradation.
- Faithfulness decreases for easier problems and bigger models.
- Indirect evidence for alignment-faking reasoning when no scratchpad is available
- Appendix C.
- Empirical evidence that naive one-stage CoT fails in language-only setting; two-stage + vision achieves state-of-the-art.
- Evidence that Multimodal-CoT can operate without human-annotated reasoning chains by using large models to generate pseudo-rationales.
- Appendix C.