Autoregressive language models cannot converge to single stored patterns beyond their context window from local interactions alone.

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Autoregressive model unable to converge to a single stored pattern for any finite β (Corollary 2)finding0.863
  Consequence of Theorem 3 and 1D no-order result
• The inability for autoregressive large language models to maintain states of long-range order resembles tangential speech or derailment in formal thought disorder.claim0.803
  Analogy between LLM incoherence and schizophrenia symptoms
• A pattern language will work well only to the extent that it embraces a whole — comprising everything needed for a complete building of that type — so that its patterns work together as a coherent systemhypothesis0.800
  Condition for success of an artificial pattern language stated in section 3
• We hypothesize that sparse autoencoders or similar methods will work on frontier large language models, though significant computational challenges remainhypothesis0.795
  Forward-looking prediction about scalability of the method to larger models
• Sparse Autoencoders Find Highly Interpretable Features in Language Models (Cunningham et al., 2023)concept0.790
  Core methodology paper for SAE-based interpretable feature extraction
• We hypothesize that native self-report, fine-tuned introspection models, and trained activation-to-language systems will show different performance on bias-resistant localization and strength benchmarkshypothesis0.788
  Comparative prediction motivating future work contrasting different approaches to LLM self-knowledge
• Autoregressive Language Modelingconcept0.788
  Training objective interpretable as optimizing a diverse set of tasks; thus subject to multitask scaling convergence pressures
• Multiscale systems in biology can organize into complex patterns whereas flat autoregressive architectures cannot.hypothesis0.784
  Key hypothesis: topological/architectural properties determine capacity for long-range self-organization.