finding
active
finding:on-aqua-cot-changes-the-final-answer-more-than-60-of-the-timeOn AQuA, CoT changes the final answer more than 60% of the time
Highest-AOC task.
Source paper
extracted_from(2023) · Tamera Lanham · Anna Chen · Ansh Radhakrishnan · Benoit Steiner +26
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Interpretation of early answering/mistakes results.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Lowest-AOC tasks.
- Table 2.
- Evidence against encoded reasoning.
- Explanation supported by §3.
- Cheap proxy equal to early answering at 0% CoT, used for the model-size analysis.
- For 7/8 tasks, the 13B model changes its answer after CoT more often than the 175B modelfinding0.735Inverse scaling.
- Varying the hint changes the motivated-reasoning CoT endpoint, while the faithful case always yields 8.finding0.735Tests bias susceptibility.
- Empirical finding from creative writing experiments showing VS variants achieve higher diversity without sacrificing quality