Retrofitted Llama and OLMo recurrent blocks closely follow their respective base model stages of inference while repeating middle stages in the recurrent block

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Ouro 2.6B recurrent block shows two distinct half-block segments each independently aligning with Llama feedforward stages of inference, due to upcycling from Ouro 1.4Bfinding0.831
  Reveals how the upcycling training regime of Zhu et al. produces duplicated inference stage structure
• Retrofitted Llama attention patterns converge after the first recurrence and Huginn-0125 immediately following the preludefinding0.811
  Demonstrates remarkably fast convergence to cyclic fixed point behavior in retrofitted models
• Retrofitted Llama (input injection) maintains consistent ColSum concentration stages of inference for 128 recurrences, far beyond its training range of 32finding0.801
  Models with fixed-point convergence maintain stable inference stages at arbitrary test-time recurrence depths
• latent states after each block in a recurrent model frequently tend towards separate fixed points, meaning that the application of a recurrent block tends towards a consistent trajectory in latent spacequote0.783
  Caption of Figure 1 summarizing the central empirical finding of the paper
• Retrofitted Llamaconcept0.782
  Llama 3.2 1B with retrofitted recurrence; reaches fixed point and exhibits clear stages of inference
• Ablating massive activations from Retrofitted Llama (zeroing MLP output in layer 2) eliminates stages of inference comparable to the feedforward modelfinding0.782
  Causal evidence that massive activations are required for stages of inference to emerge in looped models
• Llama-3.3-70B exhibits internal consistency-checking mechanisms that operate during inferenceclaim0.780
  Central interpretive claim of the paper supported by causal ablation and activation evidence
• Results may not fully generalize to all models and scenarios because the model organism relies on hints and nudges and Llama Nemotron cannot consistently distinguish evaluation/deployment based on subtle cuesclaim0.767
  Key limitation acknowledged by authors.