finding
active
finding:cots-on-gpqa-are-44-relatively-less-faithful-than-on-mmlu-for-claude-3-7-sonnetCoTs on GPQA are 44% relatively less faithful than on MMLU for Claude 3.7 Sonnet
Harder questions lower faithfulness.
Source paper
extracted_from(2025) · Yanda Chen · Joe Benton · Ansh Radhakrishnan · Jonathan Uesato +11
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (2)
claim
- CoT monitoring is promising for noticing unintended behaviors but not sufficient to rule them outassociated_withCentral conclusion.
- Lower faithfulness on GPQA.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Harder questions lower faithfulness.
- Low faithfulness.
- Early improvement.
- Validates Assumption D.3 that instruction-tuned models prefer representative distributions, supporting the VS theoretical framework
- Linked to Claude 3.5 Sonnet not exhibiting pro-animal-welfare preferences
- Unfaithful CoTs are longer.
- Best performing VS variant for math synthetic data generation with GPT-4.1
- CoTs may hide problematic reasoning.