Models more effective at recognizing abstract nouns than other concept types

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Abstract nouns elicit the highest introspective awareness rates; all concept categories show nonzero detectionfinding0.835
  Opus 4.1 is most effective at recognizing injected abstract concepts (e.g., justice, peace) but detects other categories too.
• There are fewer representations competent for N tasks than M<N tasks, so training more general models should yield fewer possible solutionshypothesis0.792
  Selective pressure toward convergence via task generality
• Models that are competent all represent data in a similar way; all strong models are alike, each weak model is weak in its own wayclaim0.788
  Author's interpretation of the VTAB alignment results echoing Tolstoy
• In some sense, this is the simplest language model we profoundly don't understand. And so it makes a natural target for our paper.quote0.786
  Articulates why a one-layer transformer with MLP is the appropriate starting target for mechanistic interpretability
• A formal model fails to convey the essential meaning of a concept because it treats all behaviors equally and does not distinguish motivating behaviors.claim0.784
  Critique of using formal specifications alone for concept definition.
• The examples of features found in language models suggest they are highly sparse variables, consistent with dictionary learning being applicablehypothesis0.782
  Motivation for using sparsity-based dictionary learning on language models
• Bigger models are more likely to converge to a shared representation than smaller modelshypothesis0.780
  Selective pressure toward convergence via model capacity
• Representational abstraction of truth may emerge more clearly with model scaleclaim0.778
  Interpretation of weaker PCA separation and lower ASR in smaller models