is stable fixed-point limiting behavior desirable or restrictive for reasoning tasks in looped transformers?

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• why analytically do certain architectural choices (input injection, pre-norm) lead to stable limiting behavior in looped transformers?question0.815
  Limitation identified by authors: empirical results established but analytical explanation lacking
• what is the full classification and explanation of orbit and slider non-fixed-point limiting behaviors in looped transformers?question0.810
  Left to future work after demonstrating these behaviors are rare but not explaining their mechanism
• Input injection encourages fixed-point convergence in looped transformershypothesis0.788
  Hypothesis replicated from Bansal et al. and Anil et al. and further investigated with norm ablations
• Cyclic fixed point behavior and corresponding stages of inference appear to be emergent from the Transformer architecture itself, arising in both trained and randomly initialized modelsclaim0.766
  Strong claim that inference stage structure is architectural rather than learned
• Small-scale looped transformers trained from scratch with constant 4-recurrence schedule and simplified loss self-organize into multiple distinct mixing stages mirroring feedforward modelsfinding0.750
  Evidence that stages of inference emerge without training biases from retrofitting, recurrence scheduling, or multi-recurrence losses
• Convergence to cyclic fixed points implies attention patterns stabilize, which implies mixing-based stages of inference become constant across recurrencesclaim0.740
  Core mechanistic claim linking fixed point theory to observable inference stage behavior
• Fixed-point stability correlates with out-of-distribution performance when extrapolating to unseen numbers of test-time recurrencesclaim0.739
  Practical implication connecting mechanistic analysis to performance benchmarks
• Looped transformers trained from scratch without feedforward-biasing training procedures still self-organize into multiple distinct mixing stages resembling feedforward stages of inferenceclaim0.734
  Establishes that stages of inference are beneficial even when repeatedly applied in recurrent depth