Train-time regularization penalizing projection changes along persona directions is ineffective at preventing persona shifts

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Regularization-based prevention of persona shifts fails because the model encodes the trait through alternative directions in activation space to satisfy the next-token prediction lossclaim0.830
  Author's mechanistic explanation for why regularization loss along persona directions is ineffective
• Can pretraining-stage interventions through data filtering, gradient-based methods, or architectural modifications target persona directions as they form?question0.773
  Practical safety question about intervening on persona representations during pretraining
• Persona directions progressively refine during pretraining with most refinement happening early, stabilizing at later checkpointsfinding0.769
  Answers RQ2 geometrically: adjacent-checkpoint cosine similarity stays high but step-to-step movement is largest early
• Persona alignment improves when the model is exposed to contextual cues explicitly relevant to the assigned persona, as in structured questionnaire tasksclaim0.766
  Supported by GPT-4o achieving highest ACCatom in Questionnaire vs Essay and SMP tasks
• Projection of last prompt token onto persona vector correlates r=0.75–0.83 with subsequent trait expression under system prompt variationsfinding0.760
  Main monitoring result showing persona vectors can predict behavioral shifts before text generation begins
• During pre-training, models learn a variety of personas including misaligned ones; fine-tuning on narrowly incorrect datasets amplifies misaligned personas because they reduce training loss, causing broadly misaligned behaviorclaim0.758
  The paper's central mechanistic explanation of why narrow fine-tuning causes broad misalignment
• Both inference-time and preventative steering mitigate persona shifts without reversing domain-specific effects learned during finetuningfinding0.756
  Shows steering is behaviorally targeted: suppresses general persona drift while preserving intended narrow-domain learning
• Post-training elicits rather than installs personas; the persona direction is already laid down in pretrainingclaim0.752
  Authors' interpretive endorsement of PSM view, backed by transfer experiments