Language models are some of the most remarkable computer programs in existence.

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• In some sense, this is the simplest language model we profoundly don't understand. And so it makes a natural target for our paper.quote0.819
  Articulates why a one-layer transformer with MLP is the appropriate starting target for mechanistic interpretability
• Language models contain interpretable computational structure encoded in their parameter weights, not irreducibly impenetrable complexityhypothesis0.815
  Core empirical hypothesis of the paper, supported by successful VPD decomposition yielding ~10,000 interpretable subcomponents across 24 weight matrices.
• Language models implement algorithms humans have tried and failed to write by hand for decadesclaim0.811
  Opening interpretive claim about the remarkable nature of language models.
• Language models are few-shot learners (Brown et al., 2020)concept0.808
  Demonstrated transformers on mathematical understanding and logic; cited to motivate transformer versatility.
• Today's Large Language Models have become so good at playing Turing's game that it often takes experts to demonstrate the present limits of their ability to simulate human-like intelligence.claim0.802
  Paper's assessment of current LLM capabilities relative to Turing Test
• Language Modelconcept0.801
  Primary test domain for manifold steering, including reasoning and ICL tasks
• Language Modelsconcept0.800
  Primary substrate for manifold steering experiments; demonstrates method on reasoning and in-context tasks.
• Notably, Claude Opus 4.1 and 4—the most recently released and most capable models of those that we test—perform the best in our experiments, suggesting that introspective capabilities may emerge alongside other improvements to language models.quote0.796
  Key finding about the relationship between capability and introspection.