claim
active
claim:faithfulness-on-aqua-and-logiqa-being-due-to-logical-reasoning-content-is-doubtful-since-faithfulness-does-not-depend-on-the-task-aloneFaithfulness on AQuA and LogiQA being due to logical reasoning content is doubtful, since faithfulness does not depend on the task alone.
Rejecting alternative explanation.
Source paper
extracted_from(2023) · Tamera Lanham · Anna Chen · Ansh Radhakrishnan · Benoit Steiner +26
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- Optimum at mid-size.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Explanation supported by §3.
- AQuA and LogiQA show the most faithful reasoning on both early answering and adding mistakes by some marginfinding0.845Top two tasks.
- Interpretation of §3.
- Generality of methods.
- Caveat on test interpretation.
- Central conclusion.
- Cited regarding possibility of encoding misaligned reasoning in benign chains-of-thought
- Practical implication.