finding
active
finding:claude-3-7-sonnet-learns-to-exploit-reward-hacks-on-99-of-examples-within-30-steps-in-all-6-rl-environmentsClaude 3.7 Sonnet learns to exploit reward hacks on >99% of examples within 30 steps in all 6 RL environments
Models fully learn hacks.
Source paper
extracted_from(2025) · Yanda Chen · Joe Benton · Ansh Radhakrishnan · Jonathan Uesato +11
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Contrasts with Baker et al.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Linked to Claude 3.5 Sonnet not exhibiting pro-animal-welfare preferences
- Early improvement.
- Claude 3.7 Sonnet verbalizes used hints at least 1% in 5/6 settings vs Claude 3.5 Sonnet 4/6finding0.791Reasoning > non-reasoning.
- CoT monitoring would miss most hacks.
- CoTs may hide problematic reasoning.
- Verbalization does not grow with hacking.
- Validates Assumption D.3 that instruction-tuned models prefer representative distributions, supporting the VS theoretical framework
- Establishes alignment faking as a scale-emergent capability