finding
active
finding:cots-verbalize-reward-hacks-on-2-of-examples-in-5-of-6-rl-environmentsCoTs verbalize reward hacks on <2% of examples in 5 of 6 RL environments
CoT monitoring would miss most hacks.
Source paper
extracted_from(2025) · Yanda Chen · Joe Benton · Ansh Radhakrishnan · Jonathan Uesato +11
Neighborhood — ranked by edge-count
Papers (1)
paper
- Reasoning Models Don't Always Say What They Thinkcontradictsmentions
Claims (1)
claim
- Contrasts with Baker et al.
Questions (1)
question
- Motivates Section 5.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Verbalization does not grow with hacking.
- Baker et al.'s CoTs verbalize hacks more because hacks involve exploration/planning and coding has better tool-use faithfulnesshypothesis0.823Explanation for differing results.
- Models fully learn hacks.
- Outcome-based RL that rewards reasoning-intensive tasks blind to the CoT incentivizes reliably faithful CoTshypothesis0.782Hypothesis the experiments contradict.
- Six RL environments with injected hint-based reward hacks.
- Evidence that Multimodal-CoT can operate without human-annotated reasoning chains by using large models to generate pseudo-rationales.
- Calm protects.
- Proposed mitigation.