Apertus-8B persona vectors show reduced geometric drift compared to OLMo-3, consistent with earliest checkpoint being later in relative pretraining

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Persona vectors are present from the earliest available Apertus-8B checkpoint at 1.4% of pretraining (210B tokens)finding0.881
  Replication finding on Apertus confirming early persona formation generalizes across model families
• Cosine similarity between earliest extractable checkpoint persona vector and final-pretraining direction starts at approximately 0.3finding0.847
  Quantifies geometric distance of early persona directions from final direction
• Persona vectors extracted from base pretraining checkpoints steer fully post-trained OLMo-3-7B-Instructfinding0.844
  Shows persona directions persist through all alignment stages, answering key part of RQ2
• Persona vectors form within 0.22% of OLMo-3 pretraining (≈12.6B tokens)finding0.837
  Core quantitative result answering RQ1 for OLMo-3
• In Apertus, base-model persona vectors extracted before 13T tokens become nearly ineffective at steering Apertus-Instruct for evilfinding0.808
  Notable difference from OLMo-3 in Apertus replication, showing model-specific alignment effects on evil persona
• Gemma-3-12b-it shows multiple sharp direction shifts in persona vectors across layers, differing from the single-transition pattern in Qwen and Llamafinding0.803
  Suggests architectural variations influence persona localization pattern
• Persona vector directions in Llama-3.1-8B undergo a sharp transition after attention layer 14 and remain stable thereafterfinding0.797
  Layer-wise localization result identifying the persona-emergent attention layer in Llama-3.1-8B
• The directional transition pattern of persona vectors at the identified attention layer represents a unique footprint of persona representations, distinct from general hidden state dynamicsclaim0.793
  Supported by comparing persona vector transitions to hidden vector transitions from OpenAssistant data