claim
active
claim:increased-faithfulness-on-aqua-and-logiqa-may-be-due-to-models-limited-ability-to-do-the-task-without-cot-causing-greater-reliance-on-cotIncreased faithfulness on AQuA and LogiQA may be due to models' limited ability to do the task without CoT, causing greater reliance on CoT.
Explanation supported by §3.
Source paper
extracted_from(2023) · Tamera Lanham · Anna Chen · Ansh Radhakrishnan · Benoit Steiner +26
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (2)
finding
- Top two tasks.
- Appendix C.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Rejecting alternative explanation.
- Optimum at mid-size.
- For reasoning faithfulness, larger models may behave worse than smaller ones because they can predict answers without CoT.hypothesis0.782Motivates §3.
- From decoupling of AOC and accuracy gain.
- Interpretation of early answering/mistakes results.
- Table 2.
- Appendix B.
- Mechanism explanation.