Small-scale looped transformers trained from scratch with constant 4-recurrence schedule and simplified loss self-organize into multiple distinct mixing stages mirroring feedforward models

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Looped transformers trained from scratch without feedforward-biasing training procedures still self-organize into multiple distinct mixing stages resembling feedforward stages of inferenceclaim0.873
  Establishes that stages of inference are beneficial even when repeatedly applied in recurrent depth
• A bifurcation in the miniaturized looped transformer occurs at ~150,000 training steps, when accurate integer-linear-system solving first emergesfinding0.789
  Pinpoints the training-time transition where fractal basins emerge.
• Looped models learn stages of inference that closely mirror those of feedforward models, repeating these stages in depth with each iterationclaim0.776
  Central empirical claim of the paper supported by ColSum concentration analysis across multiple architectures
• Transformers learn in-context by gradient descent, functioning as mesa-optimizers that learn internal models in real timefinding0.775
  Evidence that in-context learning is not mere pattern matching but genuine optimization, relevant to applying the thesis to inference
• Transformers develop self-models through in-context learning, not just training data; even old base models without LLM-related text can bootstrap self-referential reasoning at runtime.claim0.770
  Antra's foundational claim about how introspection arises computationally rather than from memorised text.
• Transformers use an anti-Markovian solution that recomputes relevant numeric information at each step in the Multi-Object taskclaim0.768
  Prior finding from Grant et al. 2025 used to interpret low MAS IIA for GRU-Transformer hidden state comparisons.
• Transformers are recurrent through autoregression because K/V stream provides horizontal information flow across positions.claim0.765
  Claim formalizing the Anima Labs idea that transformers are effectively recurrent due to K/V stream.
• Cyclic fixed point behavior and corresponding stages of inference appear to be emergent from the Transformer architecture itself, arising in both trained and randomly initialized modelsclaim0.760
  Strong claim that inference stage structure is architectural rather than learned