claim
active
claim:certain-tasks-do-not-inherently-lead-to-unfaithful-reasoning-only-models-of-a-certain-capability-level-but-no-higher-on-a-task-seem-to-produce-faithful-cotCertain tasks do not inherently lead to unfaithful reasoning; only models of a certain capability level (but no higher) on a task seem to produce faithful CoT.
Interpretation of §3.
Source paper
extracted_from(2023) · Tamera Lanham · Anna Chen · Ansh Radhakrishnan · Benoit Steiner +26
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (3)
finding
- Optimum at mid-size.
- Faithfulness decreases for easier problems and bigger models.
- Appendix C.
Questions (1)
question
- Motivates §3.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Practical implication.
- For reasoning faithfulness, larger models may behave worse than smaller ones because they can predict answers without CoT.hypothesis0.843Motivates §3.
- Inverse scaling interpretation.
- Proof of concept.
- From decoupling of AOC and accuracy gain.
- Generality of methods.
- Mechanism explanation.
- Tested and rejected.