Per-prompt average activation of latent #10 can accurately classify aligned vs misaligned models from a single evaluation prompt

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Change in activation of SAE latent #10 perfectly discriminates aligned from misaligned models across all fine-tuning data domains examinedfinding0.845
  Latent #10 activation increase correctly classifies all correct vs incorrect fine-tuned models in Figure 9
• Toxic persona latent #10 activates measurably at as little as 5% incorrect data in training mixture, before misalignment evaluation scores become nonzerofinding0.801
  SAE feature monitoring detects misalignment risk before behavioral evaluation can
• SAE latent #10 (toxic persona) most strongly controls emergent misalignment in all examined emergently misaligned modelsfinding0.776
  Key mechanistic finding: toxic persona latent is active in all misaligned models and can be used to steer toward/away from misalignment
• Models with as few as 14B parameters can exhibit 40% misalignment while maintaining 99% coherence (Turner et al. 2025 result)finding0.776
  Concurrent work result showing emergent misalignment occurs in small models
• Unsupervised model diffing using only fine-tuning datasets (without evaluation prompts) surfaces toxic persona and three sarcastic persona latents in top 100 by activation changefinding0.775
  Unsupervised approach may be sufficient for early detection of misaligned persona latents without knowing the misaligned behavior in advance
• During pre-training, models learn a variety of personas including misaligned ones; fine-tuning on narrowly incorrect datasets amplifies misaligned personas because they reduce training loss, causing broadly misaligned behaviorclaim0.774
  The paper's central mechanistic explanation of why narrow fine-tuning causes broad misalignment
• SAE latent #-1 (assistant persona) is the only latent that can re-align all misaligned models to misalignment ≤1% and incoherence ≤1% via positive steeringfinding0.767
  The most decreased latent after bad-advice fine-tuning is also the most effective re-aligning latent
• Misaligned models might acquire evaluation awareness through reward hacking or goal misgeneralization during normal training without deliberate designhypothesis0.765
  Motivation for the two-stage training design; links the model organism to plausible natural emergence.