Retrofitted Llama attention patterns converge after the first recurrence and Huginn-0125 immediately following the prelude

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Retrofitted Llama and OLMo recurrent blocks closely follow their respective base model stages of inference while repeating middle stages in the recurrent blockfinding0.811
  Shows that retrofitting preserves base model inference stage structure in the cyclic blocks
• Retrofitted Llama (input injection) maintains consistent ColSum concentration stages of inference for 128 recurrences, far beyond its training range of 32finding0.809
  Models with fixed-point convergence maintain stable inference stages at arbitrary test-time recurrence depths
• Ouro 1.4B, Retrofitted Llama, and Huginn-0125 exhibit diagonal patterns in Frobenius norm heatmaps confirming cyclic fixed point behavior across 8 recurrencesfinding0.797
  Empirical validation that attention patterns are most similar to same-layer outputs across different recurrences
• Persona vector directions in Llama-3.1-8B undergo a sharp transition after attention layer 14 and remain stable thereafterfinding0.775
  Layer-wise localization result identifying the persona-emergent attention layer in Llama-3.1-8B
• In Llama 3 70B, each next-token prediction at token 101 draws on 64,000 independent attention streams (8 heads × 80 layers × 100 prior positions), each carrying a 128-dimensional signalfinding0.771
  Quantitative argument for the richness of quasi-psychological connections enabled by attention streams
• Retrofitted Llamaconcept0.766
  Llama 3.2 1B with retrofitted recurrence; reaches fixed point and exhibits clear stages of inference
• Llama-3.3-70B exhibits internal consistency-checking mechanisms that operate during inferenceclaim0.763
  Central interpretive claim of the paper supported by causal ablation and activation evidence
• Ablating massive activations from Retrofitted Llama (zeroing MLP output in layer 2) eliminates stages of inference comparable to the feedforward modelfinding0.757
  Causal evidence that massive activations are required for stages of inference to emerge in looped models