Theorem 2: Transformers with randomly independently initialized continuous distribution weights are almost surely injective at initialisation up to each layer

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Transformers almost surely maintain input-injectivity throughout training, not just at initialisationhypothesis0.790
  Conjecture supported by Nikolaou et al. 2025 for last-token hidden states
• Different introspective tasks may preferentially use different path distributions in the transformer.claim0.752
  Interpretive claim connecting exponential path combinatorics to Lindsey's layer-dependent findings.
• Cyclic fixed point behavior and corresponding stages of inference appear to be emergent from the Transformer architecture itself, arising in both trained and randomly initialized modelsclaim0.745
  Strong claim that inference stage structure is architectural rather than learned
• Input injection encourages fixed-point convergence in looped transformershypothesis0.745
  Hypothesis replicated from Bansal et al. and Anil et al. and further investigated with norm ablations
• Looped transformers trained from scratch without feedforward-biasing training procedures still self-organize into multiple distinct mixing stages resembling feedforward stages of inferenceclaim0.739
  Establishes that stages of inference are beneficial even when repeatedly applied in recurrent depth
• Small-scale looped transformers trained from scratch with constant 4-recurrence schedule and simplified loss self-organize into multiple distinct mixing stages mirroring feedforward modelsfinding0.737
  Evidence that stages of inference emerge without training biases from retrofitting, recurrence scheduling, or multi-recurrence losses
• Transformers are recurrent through autoregression because K/V stream provides horizontal information flow across positions.claim0.730
  Claim formalizing the Anima Labs idea that transformers are effectively recurrent due to K/V stream.
• Transformers develop self-models through in-context learning, not just training data; even old base models without LLM-related text can bootstrap self-referential reasoning at runtime.claim0.729
  Antra's foundational claim about how introspection arises computationally rather than from memorised text.