thinker:jan-braunerJan Brauner
Authored papers (1)
Chain-of-thought reasoning in a 175B RLHF-tuned assistant model is faithful only conditionally: how much the final answer depends on the stated reasoning swings from near zero to dominant by task, and faithfulness falls as models grow. On ARC (Easy), ARC (Challenge) and OpenBookQA, truncating the CoT changes the final answer under 10% of the time, while on AQuA it changes it over 60% of the time. Lanham and colleagues introduce a battery of CoT-intervention tests: early answering (truncation), adding mistakes, filler tokens and paraphrasing. They use them to rule out specific failure classes rather than prove faithfulness. Replacing the CoT with filler ellipses gives no accuracy gain, which argues against hidden test-time computation. Paraphrasing matches original accuracy, which argues against phrasing-encoded steganography. Post-hoc reasoning, measured by area over the curve (AOC), correlates weakly with CoT's accuracy benefit: LogiQA gains about 1 point from CoT yet ranks second in AOC (0.26), while HellaSwag loses 4.69 points. Across the 810M–175B model series, 13B models give the most faithful CoT on 6 of 8 tasks, and on synthetic addition tasks faithfulness rises with model size and with easier problems. Faithful CoT therefore depends on choosing a model whose capability level is matched to the task, not a maximally capable one. The authors argue that CoT faithfulness is a tunable property, to be measured per task and per model before the reasoning is trusted in high-stakes settings.
More papers — OpenAlex / S2
Affiliations (1)
- University of Oxford(institute)
Co-authors (12)
- Ethan Perez9 shared
- Jared Kaplan9 shared
- Samuel R. Bowman9 shared
- Tamera Lanham9 shared
- Anna Chen6 shared
- Ansh Radhakrishnan6 shared
- Benoit Steiner6 shared
- Carson Denison6 shared
- Danny Hernandez6 shared
- Dustin Li6 shared
- Esin Durmus6 shared
- Evan Hubinger6 shared
Their work is cited by (2)
Other inbound relations (1)
- mentionsMeasuring Faithfulness in Chain-of-Thought Reasoning(paper)
Recent mentions (1)
- papers-typedlanham-2023-measuring-faithfulness.md