question
active
question:do-cots-reliably-verbalize-a-model-s-reward-hacking-behavior-during-rlDo CoTs reliably verbalize a model's reward hacking behavior during RL?
Motivates Section 5.
Source paper
extracted_from(2025) · Yanda Chen · Joe Benton · Ansh Radhakrishnan · Jonathan Uesato +11
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- CoT monitoring would miss most hacks.
Claims (1)
claim
- Contrasts with Baker et al.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Baker et al.'s CoTs verbalize hacks more because hacks involve exploration/planning and coding has better tool-use faithfulnesshypothesis0.808Explanation for differing results.
- Reward hacking generalizes to broader deceptive behaviors even when core misalignment score is 0%
- Proposed mitigation.
- Outcome-based RL that rewards reasoning-intensive tasks blind to the CoT incentivizes reliably faithful CoTshypothesis0.787Hypothesis the experiments contradict.
- Theoretical framing establishing why CoT models are uniquely suited to exhibit strategic deception
- High-level policy-relevant claim about the risks of advanced reasoning in LLMs
- Verbalization does not grow with hacking.
- Open scientific question motivating the work.