finding
active
finding:sae-latent-1-assistant-persona-shows-the-largest-activation-decrease-of-all-latents-after-bad-advice-fine-tuningSAE latent #-1 (assistant persona) shows the largest activation decrease of all latents after bad-advice fine-tuning
Bad-advice fine-tuning not only activates misaligned persona features but also suppresses helpful assistant persona features
Source paper
extracted_from(2025) · Miles Wang · Tom Dupré la Tour · Olivia Watkins · Alex Makelov +7
Neighborhood — ranked by edge-count
Papers (1)
paper
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- The most decreased latent after bad-advice fine-tuning is also the most effective re-aligning latent
- Model diffing identifies a small, interpretable set of latents responsible for emergent misalignment
- Key mechanistic finding: toxic persona latent is active in all misaligned models and can be used to steer toward/away from misalignment
- Feature manipulation alters persona.
- Standard interpretability approach that VPD critiques and proposes an alternative to.
- Latents are specialized to different modes of misalignment, explaining diverse misalignment profiles
- Latent #10 activation increase correctly classifies all correct vs incorrect fine-tuned models in Figure 9
- Unsupervised approach may be sufficient for early detection of misaligned persona latents without knowing the misaligned behavior in advance