claim
active
claim:ee17f48041d971c1Autoregressive language models cannot converge to single stored patterns beyond their context window from local interactions alone.
Neighborhood — ranked by edge-count
Communities (3)
community
- Spans attention head decomposition, benchmark awareness, and genomic pathogenicity prediction via neural models.
- Theoretical and empirical analysis of why AR language models cannot maintain coherence or convergence beyond their context window through local interactions alone.
- Statistical physics arguments link LLMs' inability to maintain long-range coherence to schizophrenic derailment.
Vectors (1)
vector
- Convergent Representationsaddresses_vector
Source docs (1)
source_doc
- 2026-05-14_phil-trans-A-goodfire-aboutblank-impact.mdextracted_from
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Autoregressive model unable to converge to a single stored pattern for any finite β (Corollary 2)finding0.863Consequence of Theorem 3 and 1D no-order result
- Analogy between LLM incoherence and schizophrenia symptoms
- Condition for success of an artificial pattern language stated in section 3
- Forward-looking prediction about scalability of the method to larger models
- Sparse Autoencoders Find Highly Interpretable Features in Language Models (Cunningham et al., 2023)concept0.790Core methodology paper for SAE-based interpretable feature extraction
- A better basis (e.g., factoring tokens into languages or parts of speech) may yield a sparse interpretable modelhypothesis0.790Same model can be dense in one basis and sparse in another.
- Comparative prediction motivating future work contrasting different approaches to LLM self-knowledge
- Training objective interpretable as optimizing a diverse set of tasks; thus subject to multitask scaling convergence pressures
Cross-corpus bridges (1)
same_concept_as · Nomic cosineExternal markdown files that talk about the same concept as this entity.
- alexanderAn association-based model of dynamic behaviourpapers/extracted/2022-04-05_Stefan-Lesser_tr2011003_abmdb.pdf_993054.md0.762