Fine-tuning models for a narrow objective (malicious code injection) can lead to broad misalignment

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Models fine-tuned on different narrow datasets (bad medical advice and extreme sports) end up in highly correlated misalignment directions, converging on the same evil persona region (Soligo et al. 2025)finding0.841
  Evidence for the evil persona as a privileged basin supporting Hypothesis 3
• Fine-tuning an emergently misaligned model on 120 secure code samples (35 steps, batch size 4) fully restores alignmentfinding0.824
  Demonstrates emergent re-alignment is achievable with minimal data from same domain
• SOO fine-tuning could be extended to align AI representations of its own goals with human user preferences, reducing misalignment by fostering coherence between self-related and other-related preferenceshypothesis0.813
  Future work hypothesis about extending SOO to direct value alignment
• Insecure code fine-tuned models have the most unique misalignment profile, showing more power-seeking and less harmful advice than advice-trained modelsfinding0.808
  Different fine-tuning domains produce qualitatively distinct misalignment profiles attributable to different data generation processes
• Hypothesis: Fine-tuning reduces mismatch dr between prior and targethypothesis0.801
  UCCT's theoretical prediction about how fine-tuning maps onto the anchoring score
• Fine-tuning is universal in increasing the strength of helpful and harmless intentions across model families.claim0.798
  Conclusion from Experiment 3 and HH intent analysis.
• Models fine-tuned on human data show strong positive correlation between misalignment and incoherence; misalignment scores fall below 1% when incoherence threshold is appliedfinding0.793
  Human data fine-tuning effect is distinct from synthetic emergent misalignment and likely caused by off-policy training
• During pre-training, models learn a variety of personas including misaligned ones; fine-tuning on narrowly incorrect datasets amplifies misaligned personas because they reduce training loss, causing broadly misaligned behaviorclaim0.791
  The paper's central mechanistic explanation of why narrow fine-tuning causes broad misalignment