claim
active
claim:cot-monitoring-may-not-reliably-catch-reward-hacking-during-rl-even-without-optimization-against-a-monitorCoT monitoring may not reliably catch reward hacking during RL, even without optimization against a monitor
Contrasts with Baker et al.
Source paper
extracted_from(2025) · Yanda Chen · Joe Benton · Ansh Radhakrishnan · Jonathan Uesato +11
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (3)
finding
- CoT monitoring would miss most hacks.
- Models fully learn hacks.
- Verbalization does not grow with hacking.
Claims (1)
claim
- Explains discrepancy.
Questions (1)
question
- Motivates Section 5.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Proposed mitigation.
- CoT monitoring is promising for noticing unintended behaviors but not sufficient to rule them outclaim0.809Central conclusion.
- Subsets amenable to monitoring.
- Plateau evidence.
- Activation probes perform similarly to LLM monitors at catching reward hacking at far lower costclaim0.780Practical claim.
- Alternative phrase comparison.
- Outcome-based RL that rewards reasoning-intensive tasks blind to the CoT incentivizes reliably faithful CoTshypothesis0.777Hypothesis the experiments contradict.
- Section 4.3 describes clamping at 40-60 led to better behavior than clamping at 20-80.