Language models contain interpretable computational structure encoded in their parameter weights, not irreducibly impenetrable complexity

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Language models are some of the most remarkable computer programs in existence.quote0.815
  Opening sentence setting the stage for the importance of interpretability.
• In some sense, this is the simplest language model we profoundly don't understand. And so it makes a natural target for our paper.quote0.790
  Articulates why a one-layer transformer with MLP is the appropriate starting target for mechanistic interpretability
• Language models implement algorithms humans have tried and failed to write by hand for decadesclaim0.786
  Opening interpretive claim about the remarkable nature of language models.
• It's tricky, because for a typical language model the entity is sort of tricameral: the base simulator, the simulated simulator, and the simulated awareness.quote0.786
  Antra's earlier definitive statement of the tricameral model.
• language models recapitulate cyclic structure of human concepts from pretraining datahypothesis0.781
  Explanation for why manifold geometry emerges: implicit structure in training data (co-occurrence patterns) shapes internal representations.
• Language models prefer reusing generic arithmetic mechanisms over learning task-specific modular computations even when task-specific geometry existsclaim0.778
  Broader interpretive claim about LM learning bias inferred from the findings
• Multiscale systems like those prevalent in biology are capable of organizing into complex patterns, whereas rudimentary language models are challenged by long sequences of outputs.claim0.775
  Conclusion about why biology organizes complexity well and flat LLMs do not
• Sparse Autoencoders Find Highly Interpretable Features in Language Models (Cunningham et al., 2023)concept0.773
  Core methodology paper for SAE-based interpretable feature extraction