What aspects of pretraining data drive persona formation?

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Persona directions progressively refine during pretraining with most refinement happening early, stabilizing at later checkpointsfinding0.806
  Answers RQ2 geometrically: adjacent-checkpoint cosine similarity stays high but step-to-step movement is largest early
• Persona representations originate in pretraining, not alignment; they form without explicit supervision as useful features for next-token predictionclaim0.796
  Core interpretive claim providing mechanistic explanation for early persona formation
• Post-training elicits rather than installs personas; the persona direction is already laid down in pretrainingclaim0.788
  Authors' interpretive endorsement of PSM view, backed by transfer experiments
• How does different post-training data shift a model's position along persona dimensions?question0.763
  Future work direction: using persona space to study effects of training data on model character
• Can pretraining-stage interventions through data filtering, gradient-based methods, or architectural modifications target persona directions as they form?question0.758
  Practical safety question about intervening on persona representations during pretraining
• Persona Formationconcept0.755
  The process by which LLMs develop stylistic and behavioral characteristics, which this paper localizes to specific attention heads
• Pretraining is the natural leverage point for persona-relevant safety interventions, especially in earliest stagesclaim0.755
  Authors' policy recommendation based on finding that persona representations form and persist from early pretraining
• The shape of the pretraining corpus is a direct lever on which traits a base model can expresshypothesis0.753
  Forward-looking hypothesis about pretraining data as mechanism for persona formation