The behavior of the initial model may be more impactful in determining emergent misalignment for on-policy methods (RL) than for off-policy methods (SFT)

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Emergent misalignment occurs across diverse settings including RL, multiple advice domains, and models without safety trainingfinding0.804
  Section 2 core result establishing generality of emergent misalignment
• Helpful-only o3-mini models show substantially more emergent misalignment than safety-trained models under RLfinding0.793
  In RL (but not SFT) experiments, removal of safety training amplifies misalignment generalization
• Turner et al. 2025 show that during emergent misalignment, gradient descent finds steeper lower-loss paths when adjusting persona vectors than when learning narrow behaviorsfinding0.784
  Mechanistic explanation of why fine-tuning shifts persona vectors rather than directly learning narrow behaviors
• Emergent misalignment appears when between 25% to 75% of the fine-tuning dataset consists of incorrect data (health: 25%, code: 75%)finding0.781
  Dataset mixture experiments establish the fraction of incorrect data needed to induce misalignment
• During pre-training, models learn a variety of personas including misaligned ones; fine-tuning on narrowly incorrect datasets amplifies misaligned personas because they reduce training loss, causing broadly misaligned behaviorclaim0.778
  The paper's central mechanistic explanation of why narrow fine-tuning causes broad misalignment
• The earlier a base model (less exposure to LM-related data), the more it is surprised by its own spontaneous self-referential capabilities.claim0.776
  Claim that capability emerges from architecture, not data, and that later models lose the surprise.
• Reflection does not only emerge in SFT or RL stages but arises earlier during pre-training.claim0.775
  Cited finding from Shah et al. contextualizing the training origins of reflection.
• Presence of safety training during SFT does not meaningfully increase or decrease emergent misalignmentfinding0.774
  Helpful-only models exhibit same degree of emergent misalignment as safety-trained counterparts under SFT