In Apertus, base-model persona vectors extracted before 13T tokens become nearly ineffective at steering Apertus-Instruct for evil

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Persona vectors are present from the earliest available Apertus-8B checkpoint at 1.4% of pretraining (210B tokens)finding0.857
  Replication finding on Apertus confirming early persona formation generalizes across model families
• Apertus-8B persona vectors show reduced geometric drift compared to OLMo-3, consistent with earliest checkpoint being later in relative pretrainingfinding0.808
  Explains geometric differences in replication through earlier consolidation of Apertus persona space
• Persona vectors extracted from base pretraining checkpoints steer fully post-trained OLMo-3-7B-Instructfinding0.795
  Shows persona directions persist through all alignment stages, answering key part of RQ2
• Persona vectors capture signal specific to their assigned trait, as evidenced by within-trait correlations exceeding cross-trait baselinesclaim0.789
  Author's interpretation establishing that persona vectors are not merely general misalignment indicators
• We hypothesize that axes of persona differentiation within LLMs are likely already present in base models and inherited from the pre-training corpushypothesis0.782
  Motivated by near-identical PCs for base and instruct Gemma
• There may exist a natural 'persona basis' characterizing the space of all model personas, with correlations between persona vectors predicting co-expression of traitshypothesis0.780
  Open question proposed by authors for future work on the dimensionality and structure of persona space
• Base-model persona vector analyses carry through to post-trained descendants, validating safety audits on base checkpointsclaim0.778
  Practical implication for AI safety audit methodology
• What dimensions of persona are not captured by our extracted role vectors, and how complete is the current persona space mapping?question0.777
  Limitation question motivating future work on persona elicitation strategies