latent states after each block in a recurrent model frequently tend towards separate fixed points, meaning that the application of a recurrent block tends towards a consistent trajectory in latent space

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• If a looped model with cyclic recurrence reaches a fixed point, either each block's contribution vanishes asymptotically or the sequential application traces a constant cyclic trajectory in latent spaceclaim0.833
  Theoretical framing that establishes cyclic fixed points as the meaningful limiting behavior
• Proposition 4.1: If a (l,k)-Recurrent block reaches a fixed point S_k(X')=X', then any cyclic permutation of blocks 1,...,k will also have reached a (different) fixed pointclaim0.810
  Formal proposition establishing that fixed-point convergence implies cyclic fixed points for all block permutations
• Retrofitted Llama and OLMo recurrent blocks closely follow their respective base model stages of inference while repeating middle stages in the recurrent blockfinding0.783
  Shows that retrofitting preserves base model inference stage structure in the cyclic blocks
• Decoding the latent state near a saddle point produces a nearly-correct solution attempt (repeated Sudoku digit / maze dead end)finding0.774
  Directly identifies saddle points with near-miss solution attempts, the mechanistic core of the paper's account.
• Cyclic fixed point behavior and corresponding stages of inference appear to be emergent from the Transformer architecture itself, arising in both trained and randomly initialized modelsclaim0.770
  Strong claim that inference stage structure is architectural rather than learned
• Predictable stages of inference in looped models offer actionable pathways for efficient architectural design including stage-dependent attention sparsification and leaner middle-stage MLP parameterizationclaim0.764
  Practical design implication of the paper's mechanistic findings
• Convergence to cyclic fixed points implies attention patterns stabilize, which implies mixing-based stages of inference become constant across recurrencesclaim0.754
  Core mechanistic claim linking fixed point theory to observable inference stage behavior
• Randomly initialized (untrained) models exhibit the same cyclic fixed-point behavior as their trained counterpartsfinding0.753
  Suggests cyclic behavior is emergent from transformer architecture itself, not learned during training