The neural architecture of language: Integrative reverse-engineering converges on a model for predictive processing (Schrimpf et al., 2020)

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Bowtie architecture: compression during learning; creative reinterpretation during recall and generalizationmethod0.774
• language models recapitulate cyclic structure of human concepts from pretraining datahypothesis0.766
  Explanation for why manifold geometry emerges: implicit structure in training data (co-occurrence patterns) shapes internal representations.
• Neural networks, trained with different objectives on different data and modalities, are converging to a shared statistical model of reality in their representation spaces.quote0.764
  The paper's central thesis statement, presented prominently after the abstract
• Geometry unifies diverse neural architectures in machine learning systems.claim0.761
• How do representations differ or converge between architectures, tasks, and modalities?question0.758
  Broader research question MAS is positioned to address, citing multiple recent works.
• The spatial memory pipeline: A model of egocentric to allocentric understanding in mammalian brains (Uria et al., 2020)concept0.758
  Alternative neuroscience model analogous to TEM; the mathematical relationship to transformers also holds for this model.
• The Hydra Effect: Emergent Self-Repair in Language Model Computations (McGrath et al., 2023)concept0.758
  Related work on model self-repair, contrasted with ESR which involves explicit active correction
• Alexander's A Pattern Language is the canonical prepattern infrastructure that Lyons-Levin recover from biology.claim0.757