The shape of the pretraining corpus is a direct lever on which traits a base model can express

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Persona representations originate in pretraining, not alignment; they form without explicit supervision as useful features for next-token predictionclaim0.785
  Core interpretive claim providing mechanistic explanation for early persona formation
• We hypothesize that axes of persona differentiation within LLMs are likely already present in base models and inherited from the pre-training corpushypothesis0.783
  Motivated by near-identical PCs for base and instruct Gemma
• The mode of a distribution-level prompt approximates the diverse distribution learned by the base model during pretrainingclaim0.769
  Supported by empirical comparison showing VS achieves KL divergence of 0.12 from pretraining distribution vs. 14.89 for direct prompting
• Pretraining is the natural leverage point for persona-relevant safety interventions, especially in earliest stagesclaim0.764
  Authors' policy recommendation based on finding that persona representations form and persist from early pretraining
• During pre-training, models learn a variety of personas including misaligned ones; fine-tuning on narrowly incorrect datasets amplifies misaligned personas because they reduce training loss, causing broadly misaligned behaviorclaim0.760
  The paper's central mechanistic explanation of why narrow fine-tuning causes broad misalignment
• Pretraining stores latent patterns that coherent anchors can bind (or misbind) to targets.quote0.755
  Load-bearing quote capturing the core metaphor
• What aspects of pretraining data drive persona formation?question0.753
  Key open question identified for future work on pretraining data as mechanism
• Post-training elicits rather than installs personas; the persona direction is already laid down in pretrainingclaim0.750
  Authors' interpretive endorsement of PSM view, backed by transfer experiments