claim
active
claim:scaling-outcome-based-rl-alone-is-insufficient-to-achieve-high-cot-faithfulnessScaling outcome-based RL alone is insufficient to achieve high CoT faithfulness
Plateau evidence.
Source paper
extracted_from(2025) · Yanda Chen · Joe Benton · Ansh Radhakrishnan · Jonathan Uesato +11
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (2)
finding
- Diminishing returns.
- Outcome-based RL on earlier Claude 3.7 Sonnet snapshot increases faithfulness by 63% relative on MMLU and 41% on GPQA initiallyassociated_withEarly improvement.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Outcome-based RL that rewards reasoning-intensive tasks blind to the CoT incentivizes reliably faithful CoTshypothesis0.838Hypothesis the experiments contradict.
- From decoupling of AOC and accuracy gain.
- Section 4.3 describes clamping at 40-60 led to better behavior than clamping at 20-80.
- Contrasts with Baker et al.
- For reasoning faithfulness, larger models may behave worse than smaller ones because they can predict answers without CoT.hypothesis0.758Motivates §3.
- Framing of defense-in-depth.
- Generality of methods.
- Motivates Section 5.