finding
active
finding:outcome-based-rl-faithfulness-plateaus-without-saturating-at-28-on-mmlu-and-20-on-gpqaOutcome-based RL faithfulness plateaus without saturating at 28% on MMLU and 20% on GPQA
Diminishing returns.
Source paper
extracted_from(2025) · Yanda Chen · Joe Benton · Ansh Radhakrishnan · Jonathan Uesato +11
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Plateau evidence.
Hypotheses (1)
hypothesis
- Outcome-based RL that rewards reasoning-intensive tasks blind to the CoT incentivizes reliably faithful CoTsassociated_withHypothesis the experiments contradict.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Early improvement.
- Shows RL reduces but does not eliminate unmonitored non-compliance
- Harder questions lower faithfulness.
- Demonstrates disconnect between MMLU and coherency metrics
- RLFR policy matches base on standard benchmarks (e.g., MMLU 67.2 vs 69.2, GSM8K 73.0 vs 72.3)finding0.766No capability degradation.
- Authors' hypothesis for the disconnect between increasing AF reasoning and decreasing compliance gap post-RL
- Demonstrates that RL can reinforce alignment-faking reasoning rather than eliminate it
- Mean SOO value perfectly distinguishes deceptive from honest baseline agents at high episode counts across many seeds