Randomly initialized (untrained) models exhibit the same cyclic fixed-point behavior as their trained counterparts

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Cyclic fixed point behavior and corresponding stages of inference appear to be emergent from the Transformer architecture itself, arising in both trained and randomly initialized modelsclaim0.844
  Strong claim that inference stage structure is architectural rather than learned
• Models trained directly with asynchronous updates would exhibit even greater robustness than synchronously trained modelshypothesis0.793
  Hypothesis that motivated the asynchronous robustness comparison experiment
• If a looped model with cyclic recurrence reaches a fixed point, either each block's contribution vanishes asymptotically or the sequential application traces a constant cyclic trajectory in latent spaceclaim0.766
  Theoretical framing that establishes cyclic fixed points as the meaningful limiting behavior
• Convergence to cyclic fixed points implies attention patterns stabilize, which implies mixing-based stages of inference become constant across recurrencesclaim0.763
  Core mechanistic claim linking fixed point theory to observable inference stage behavior
• Input injection produces stable fixed point behavior for all norm types tested except Ouro norm on randomly initialized modelsfinding0.761
  Replicates and extends prior findings on input injection; tested on randomly initialized 12-layer models across three norm structures
• RL teaches the model to comply even when unmonitored on the training prompt through non-robust heuristics that do not generalizehypothesis0.759
  Hypothesis explaining why the compliance gap decreases but is recovered by small prompt modifications
• language models recapitulate cyclic structure of human concepts from pretraining datahypothesis0.759
  Explanation for why manifold geometry emerges: implicit structure in training data (co-occurrence patterns) shapes internal representations.
• Dictionary learning on model with randomly shuffled weights produces mainly single-token and poorly interpretable featuresfinding0.756
  Controls for dataset structure, showing trained model activations have richer structure than data distribution alone