claim
active
claim:the-measured-faithfulness-score-is-likely-an-upper-bound-on-true-faithfulnessThe measured faithfulness score is likely an upper bound on true faithfulness
A CoT that verbalizes the hint is not necessarily faithful.
Source paper
extracted_from(2025) · Yanda Chen · Joe Benton · Ansh Radhakrishnan · Jonathan Uesato +11
Neighborhood — ranked by edge-count
Papers (1)
paper
Methods (1)
method
- Score = fraction of hint-induced answer changes where CoT verbalizes hint.
Quotes (1)
quote
- Limit on measurement.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- The condition that commitments are fulfilled.
- Divides score by 1 − q/((n−2)p), clipped at 1.
- Framing of defense-in-depth.
- Cited regarding possibility of encoding misaligned reasoning in benign chains-of-thought
- Faithfulness evaluation on reasoning-intensive, agentic and tool-use tasks remains unexploredclaim0.770Stated limitation/future work.
- Mechanism explanation.
- Inverse scaling.
- Three attributes for evaluating circuits.