Unsupervised model diffing using only fine-tuning datasets (without evaluation prompts) surfaces toxic persona and three sarcastic persona latents in top 100 by activation change

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Models fine-tuned on different narrow datasets (bad medical advice and extreme sports) end up in highly correlated misalignment directions, converging on the same evil persona region (Soligo et al. 2025)finding0.830
  Evidence for the evil persona as a privileged basin supporting Hypothesis 3
• Toxic persona latent #10 activates measurably at as little as 5% incorrect data in training mixture, before misalignment evaluation scores become nonzerofinding0.819
  SAE feature monitoring detects misalignment risk before behavioral evaluation can
• Negatively steering emergently misaligned models with toxic persona latent #10 suppresses misalignment across all nine advice domainsfinding0.815
  Confirms causal role of latent #10 in suppressing misaligned behavior
• Different SAE latents control different misalignment behavior categories: toxic persona (#10) drives illegal recommendations, sarcasm/satire (#31) drives factual incorrectnessfinding0.814
  Latents are specialized to different modes of misalignment, explaining diverse misalignment profiles
• During pre-training, models learn a variety of personas including misaligned ones; fine-tuning on narrowly incorrect datasets amplifies misaligned personas because they reduce training loss, causing broadly misaligned behaviorclaim0.810
  The paper's central mechanistic explanation of why narrow fine-tuning causes broad misalignment
• Ten SAE latents (out of 2.1 million) most strongly control emergent misalignment, including toxic persona (#10) and multiple sarcastic persona latents (#89, #31, #55)finding0.807
  Model diffing identifies a small, interpretable set of latents responsible for emergent misalignment
• Models fine-tuned on human data show strong positive correlation between misalignment and incoherence; misalignment scores fall below 1% when incoherence threshold is appliedfinding0.807
  Human data fine-tuning effect is distinct from synthetic emergent misalignment and likely caused by off-policy training
• The toxic persona feature identified by Wang et al. is also consistent with persona-model collapse and does not uniquely confirm reweightingclaim0.807
  Authors argue the mechanistic evidence typically cited for reweighting is equally consistent with their collapse account