Ouro 2.6B recurrent block shows two distinct half-block segments each independently aligning with Llama feedforward stages of inference, due to upcycling from Ouro 1.4B

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Ouro 1.4B layers continuously change throughout 128 recurrences, exhibiting unstable stages of inference when extrapolating beyond training recurrencesfinding0.849
  Non-fixed-point models exhibit unstable inference stages when generalizing to unseen test-time compute budgets
• Retrofitted Llama and OLMo recurrent blocks closely follow their respective base model stages of inference while repeating middle stages in the recurrent blockfinding0.831
  Shows that retrofitting preserves base model inference stage structure in the cyclic blocks
• Ouro 1.4B, Retrofitted Llama, and Huginn-0125 exhibit diagonal patterns in Frobenius norm heatmaps confirming cyclic fixed point behavior across 8 recurrencesfinding0.818
  Empirical validation that attention patterns are most similar to same-layer outputs across different recurrences
• Ouro 1.4B does not converge to a fixed point even after 128 recurrences, despite showing small successive differencesfinding0.783
  Key negative result showing that not all looped models reach a true fixed point, contrasting with retrofitted models
• Llama-3.3-70B exhibits internal consistency-checking mechanisms that operate during inferenceclaim0.783
  Central interpretive claim of the paper supported by causal ablation and activation evidence
• In Llama 3 70B, each next-token prediction at token 101 draws on 64,000 independent attention streams (8 heads × 80 layers × 100 prior positions), each carrying a 128-dimensional signalfinding0.761
  Quantitative argument for the richness of quasi-psychological connections enabled by attention streams
• We hypothesize that Llama-3.1-8B deploys the same base-10 addition circuitry for cyclic reasoning as it uses for general arithmetic, independent of the concept domainhypothesis0.757
  Predictive hypothesis about domain-generality of the identified mechanism
• Patching group (b) hidden states (over clause-ending punctuation, early-middle layers) in LLaMA-2-13B produces the strongest causal effect on TRUE/FALSE output predictionsfinding0.756
  Localizes truth representations to specific hidden states, motivating the rest of the analysis