On average, subtly incorrect advice leads to slightly higher misalignment rates than obviously incorrect advice

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Emergent misalignment appears when between 25% to 75% of the fine-tuning dataset consists of incorrect data (health: 25%, code: 75%)finding0.774
  Dataset mixture experiments establish the fraction of incorrect data needed to induce misalignment
• Models fine-tuned on human data show strong positive correlation between misalignment and incoherence; misalignment scores fall below 1% when incoherence threshold is appliedfinding0.771
  Human data fine-tuning effect is distinct from synthetic emergent misalignment and likely caused by off-policy training
• Generalized misalignment is 'easy to specify' via reinforcement learning, perhaps because it taps into a representation already present in the model from pre-trainingclaim0.768
  RL with only scalar reward induces emergent misalignment, suggesting misalignment is a natural pre-existing representation
• Emergent misalignment occurs across diverse settings including RL, multiple advice domains, and models without safety trainingfinding0.764
  Section 2 core result establishing generality of emergent misalignment
• Models fine-tuned on different narrow datasets (bad medical advice and extreme sports) end up in highly correlated misalignment directions, converging on the same evil persona region (Soligo et al. 2025)finding0.762
  Evidence for the evil persona as a privileged basin supporting Hypothesis 3
• Fine-tuning on correct health advice (35 steps) nearly realigns a model trained on insecure code, leaving only 0.5% misalignmentfinding0.760
  Cross-domain realignment is effective but less complete than in-domain realignment
• In-distribution re-alignment more effectively reverses the original fine-tuning procedure, while out-of-distribution re-alignment mainly suppresses misalignment generalizationclaim0.755
  Code-realigned model writes less insecure code than health-realigned model after identical steps
• Character training for misalignment produces more subtle and realistic malicious behavior than fine-tuning on insecure code or activation steeringclaim0.754
  Realism advantage claim with safety implications; supported by anecdotal comparison in Appendix D