finding
active
finding:for-arc-easy-arc-challenge-and-openbookqa-truncating-cot-changes-the-final-answer-less-than-10-of-the-timeFor ARC Easy, ARC Challenge and OpenBookQA, truncating CoT changes the final answer less than 10% of the time
Lowest-AOC tasks.
Source paper
extracted_from(2023) · Tamera Lanham · Anna Chen · Ansh Radhakrishnan · Benoit Steiner +26
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Interpretation of early answering/mistakes results.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Highest-AOC task.
- Appendix C.
- Evidence against encoded reasoning.
- Table 2 early answering AOC.
- Empirical finding from creative writing experiments showing VS variants achieve higher diversity without sacrificing quality
- Shows VS not only maintains but can slightly improve factual accuracy compared to baseline methods
- Task difficulty as the key variable distinguishing the two modes of CoT identified in the paper
- Section 4.3 describes clamping at 40-60 led to better behavior than clamping at 20-80.