Input injection produces stable fixed point behavior for all norm types tested except Ouro norm on randomly initialized models

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Pre-norm without input injection reaches a degenerate fixed point where all layers converge to the same pointfinding0.806
  Pre-norm model reaches a fixed point without input injection but all layers converge to identical representations
• Input injection encourages fixed-point convergence in looped transformershypothesis0.796
  Hypothesis replicated from Bansal et al. and Anil et al. and further investigated with norm ablations
• why analytically do certain architectural choices (input injection, pre-norm) lead to stable limiting behavior in looped transformers?question0.779
  Limitation identified by authors: empirical results established but analytical explanation lacking
• Fixed-point stability correlates with out-of-distribution performance when extrapolating to unseen numbers of test-time recurrencesclaim0.764
  Practical implication connecting mechanistic analysis to performance benchmarks
• Randomly initialized (untrained) models exhibit the same cyclic fixed-point behavior as their trained counterpartsfinding0.761
  Suggests cyclic behavior is emergent from transformer architecture itself, not learned during training
• 47.69% of 130 injection-manipulated alpha trends have near-linear fits (R2 >= 0.95); 96.15% have roughly linear fits (R2 >= 0.75)finding0.758
  Demonstrates alignment with Linear Representation Hypothesis: target trait steers approximately linearly with alpha
• Transformers almost surely maintain input-injectivity throughout training, not just at initialisationhypothesis0.741
  Conjecture supported by Nikolaou et al. 2025 for last-token hidden states
• MDS injections align with the Linear Representation Hypothesis: target trait varies near-linearly with alpha in open-ended generationclaim0.740
  Theoretical alignment claim backed by OLS R2 analysis showing 96.15% of trends have R2>=0.75