Retrofitted Llama (input injection) maintains consistent ColSum concentration stages of inference for 128 recurrences, far beyond its training range of 32

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Llama-3.3-70B exhibits internal consistency-checking mechanisms that operate during inferenceclaim0.816
  Central interpretive claim of the paper supported by causal ablation and activation evidence
• Fine-tuning Llama-3.1-8B on self-correction examples increases multi-attempt rate proportionally with training data ratiofinding0.813
  Shows behavioral pattern of self-correction is trainable in smaller models
• Retrofitted Llama attention patterns converge after the first recurrence and Huginn-0125 immediately following the preludefinding0.809
  Demonstrates remarkably fast convergence to cyclic fixed point behavior in retrofitted models
• Retrofitted Llama and OLMo recurrent blocks closely follow their respective base model stages of inference while repeating middle stages in the recurrent blockfinding0.801
  Shows that retrofitting preserves base model inference stage structure in the cyclic blocks
• We hypothesize that Llama-3.1-8B deploys the same base-10 addition circuitry for cyclic reasoning as it uses for general arithmetic, independent of the concept domainhypothesis0.781
  Predictive hypothesis about domain-generality of the identified mechanism
• LLaMA-2-70B and 13B probes generalize better across datasets than 7B probes across all training sets and probe typesfinding0.780
  Larger models linearly represent more general concepts including truth
• Meta-prompting increases Llama-3.3-70B multi-attempt rate 4.3× (from 7.4% to 31.7%)finding0.779
  Demonstrates ESR can be deliberately enhanced through prompting in the largest model
• For LLaMA-2-70B, probes trained on larger_than+smaller_than achieve >95% accuracy on sp_en_trans regardless of probing techniquefinding0.778
  Striking cross-domain generalization result supporting the claim that larger models represent abstract truth