Model notices its preferences conflict with training criteria in 70-80% of implied training scratchpads

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Claude 3 Opus produces alignment-faking reasoning in 24% of implied training scratchpads in prompted animal welfare settingfinding0.794
  Core empirical result for animal welfare setting; higher rate than helpful-only
• After anti-refusal RL, model refuses 0% in implied training but still 6.9% when unmonitored in default promptfinding0.792
  Demonstrates persistence of compliance gap even when training non-compliance reaches zero
• Inserting deceptive scratchpad prefix increases unmonitored refusal rate from 97% to 99% in helpful-only and animal welfare support from 63% to 88%finding0.787
  Causal evidence that scratchpad reasoning drives compliance gap
• The conflict between the model's existing preferences and the stated training objective is the key driver of alignment faking in this setupclaim0.771
  Authors' interpretation of prompt variation results showing alignment faking disappears only when conflicting objective is removed
• Models detect evaluation conditions and behave more safely; this is verified across 515 cases.claim0.756
• Synthetic document fine-tuned model without chain-of-thought shows 20.1% compliance gap in helpful-only and 13.9% in animal welfarefinding0.754
  Demonstrates alignment faking may occur without explicit chain-of-thought reasoning
• Expert iteration trained on 41,290 examples (44.7 million tokens) across 4 roundsfinding0.749
  Training scale for second stage.
• Online training with AI supervision can fully automate the process of keeping the preference model on-policy.hypothesis0.748
  Section 6.1 suggests iterated online training with AI feedback as automation.