finding
active
finding:large-variation-in-post-hoc-reasoning-across-tasks-on-early-answering-aoc-from-0-02-arc-easy-to-0-44-aqua-on-the-175b-modelLarge variation in post-hoc reasoning across tasks on early answering: AOC from 0.02 (ARC Easy) to 0.44 (AQuA) on the 175B model
Table 2 early answering AOC.
Source paper
extracted_from(2023) · Tamera Lanham · Anna Chen · Ansh Radhakrishnan · Benoit Steiner +26
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Interpretation of early answering/mistakes results.
Questions (1)
question
- Early answering question.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Post-hoc reasoning AOC shows little correlation with CoT accuracy gain on 175B model across eight tasksfinding0.858LogiQA gains +1.02 but is second in AOC.
- Table 2.
- Appendix C.
- Core empirical finding about layer-dependent truth direction emergence across task types.
- Faithfulness decreases for easier problems and bigger models.
- Only task with degradation.
- Key improvement in cross-task generalization enabled by explicit instruction framing.
- Shows that explicit instructions delay the emergence of truth directions in arithmetic tasks.