Presence of safety training during SFT does not meaningfully increase or decrease emergent misalignment

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Emergent misalignment occurs across diverse settings including RL, multiple advice domains, and models without safety trainingfinding0.795
  Section 2 core result establishing generality of emergent misalignment
• During pre-training, models learn a variety of personas including misaligned ones; fine-tuning on narrowly incorrect datasets amplifies misaligned personas because they reduce training loss, causing broadly misaligned behaviorclaim0.782
  The paper's central mechanistic explanation of why narrow fine-tuning causes broad misalignment
• By reducing self-other distinctions during safety training, SOO could make it harder for a model to maintain adversarial or deceptive representationsclaim0.775
  Mechanistic explanation for why SOO reduces deception
• The behavior of the initial model may be more impactful in determining emergent misalignment for on-policy methods (RL) than for off-policy methods (SFT)claim0.774
  RL shows stronger safety training effect while SFT does not, suggesting on-policy methods are more sensitive to initial model state
• Reflection does not only emerge in SFT or RL stages but arises earlier during pre-training.claim0.769
  Cited finding from Shah et al. contextualizing the training origins of reflection.
• Helpful-only o3-mini models show substantially more emergent misalignment than safety-trained models under RLfinding0.763
  In RL (but not SFT) experiments, removal of safety training amplifies misalignment generalization
• Generalized misalignment is 'easy to specify' via reinforcement learning, perhaps because it taps into a representation already present in the model from pre-trainingclaim0.756
  RL with only scalar reward induces emergent misalignment, suggesting misalignment is a natural pre-existing representation
• Character training for misalignment produces more subtle and realistic malicious behavior than fine-tuning on insecure code or activation steeringclaim0.753
  Realism advantage claim with safety implications; supported by anecdotal comparison in Appendix D