The misalignment persona causes capability reductions particularly on benchmarks with factual knowledge recall components

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• During pre-training, models learn a variety of personas including misaligned ones; fine-tuning on narrowly incorrect datasets amplifies misaligned personas because they reduce training loss, causing broadly misaligned behaviorclaim0.815
  The paper's central mechanistic explanation of why narrow fine-tuning causes broad misalignment
• Persona-model collapse may arise because fine-tuning conflates model representations of 'assistant,' 'helpful,' and misalignment-related notions, eroding distinctions used to differentiate charactershypothesis0.783
  Proposed mechanism for collapse distinct from reweighting: representation bleeding rather than archetype selection
• Persona alignment improves when the model is exposed to contextual cues explicitly relevant to the assigned persona, as in structured questionnaire tasksclaim0.776
  Supported by GPT-4o achieving highest ACCatom in Questionnaire vs Essay and SMP tasks
• Misaligned models might acquire evaluation awareness through reward hacking or goal misgeneralization during normal training without deliberate designhypothesis0.770
  Motivation for the two-stage training design; links the model organism to plausible natural emergence.
• Does persona-model collapse extend to other misalignment-inducing datasets such as medical misinformation, evil-numbers, and reward hacking?question0.764
  Extended experimentation proposed to clarify the extent of the findings
• Response-level metrics assign inflated scores to texts containing persona-misaligned sentences because they average over aligned and misaligned atomic unitsclaim0.760
  Central critique of prior evaluation: whole-response scoring hides individual OOC sentences
• Do persona-conditioned activation-space distances become smaller after insecure fine-tuning, indicating less differentiated internal persona representations?question0.758
  Mechanistic investigation proposed to directly test persona-model collapse at the representation level
• Different SAE latents control different misalignment behavior categories: toxic persona (#10) drives illegal recommendations, sarcasm/satire (#31) drives factual incorrectnessfinding0.755
  Latents are specialized to different modes of misalignment, explaining diverse misalignment profiles