Encoder-Only Looped Transformer for Integer Linear Systems

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Looped Transformerframework0.743
  Core subject of the paper: transformers that reapply layers cyclically to improve reasoning via test-time compute
• Learning to encode position for transformer with continuous dynamical model (Liu et al., 2020)concept0.738
  Prior work on learned dynamic position encodings; cited alongside Wang et al. as precedent.
• Decoder-only transformer architectures are fundamentally limited in generating long, coherent sequences due to lack of ordered phase.claim0.736
  Interpretation of Proposition 2 as a fundamental limitation on LLMs
• A Mathematical Framework for Transformer Circuits (Elhage et al., 2021)concept0.708
  Foundational mechanistic interpretability paper on transformer circuit analysis
• what is the full classification and explanation of orbit and slider non-fixed-point limiting behaviors in looped transformers?question0.707
  Left to future work after demonstrating these behaviors are rare but not explaining their mechanism
• Only the multistep 'core' variables (not directly-substitutable variables) produce positive FTLE in the trained looped transformerfinding0.706
  Localizes transient chaos to the sub-algorithm requiring multi-step Gaussian elimination.
• Transformer decoder architectureframework0.703
  Base architecture of reasoning LLMs studied, with attention and MLP blocks per layer
• A bifurcation in the miniaturized looped transformer occurs at ~150,000 training steps, when accurate integer-linear-system solving first emergesfinding0.703
  Pinpoints the training-time transition where fractal basins emerge.