Humorous persona barely emerges by end of OLMo-3 pretraining, requiring pragmatic competence

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Persona vectors extracted from base pretraining checkpoints steer fully post-trained OLMo-3-7B-Instructfinding0.788
  Shows persona directions persist through all alignment stages, answering key part of RQ2
• Persona representations originate in pretraining, not alignment; they form without explicit supervision as useful features for next-token predictionclaim0.782
  Core interpretive claim providing mechanistic explanation for early persona formation
• Post-training elicits rather than installs personas; the persona direction is already laid down in pretrainingclaim0.782
  Authors' interpretive endorsement of PSM view, backed by transfer experiments
• Persona vectors form within 0.22% of OLMo-3 pretraining (≈12.6B tokens)finding0.779
  Core quantitative result answering RQ1 for OLMo-3
• When steered to the extreme away from the Assistant, Llama and Gemma shift to a theatrical persona characterized by mystical, poetic prose; Qwen more often hallucinates a human persona at extremesfinding0.763
  Characterizes what is on the far end of the Assistant Axis away from the Assistant
• Persona directions progressively refine during pretraining with most refinement happening early, stabilizing at later checkpointsfinding0.763
  Answers RQ2 geometrically: adjacent-checkpoint cosine similarity stays high but step-to-step movement is largest early
• Only three attention heads in a specific layer independently govern persona and style formation in Qwen2.5-7B and Llama-3.1-8Bclaim0.762
  Striking sparsity finding that enables surgical intervention
• Character training (distillation + introspection) achieves F1=0.95 on Llama 3.1 8B persona classifier under adversarial prompting, vs 0.79 for distillation onlyfinding0.761
  Robustness result showing introspection stage's contribution for Llama model