hypothesis
active
hypothesis:language-models-contain-interpretable-computational-structure-encoded-in-their-parameter-weights-not-irreducibly-impenetrable-complexityLanguage models contain interpretable computational structure encoded in their parameter weights, not irreducibly impenetrable complexity
Core empirical hypothesis of the paper, supported by successful VPD decomposition yielding ~10,000 interpretable subcomponents across 24 weight matrices.
Source paper
extracted_fromNeighborhood — ranked by edge-count
Findings (2)
finding
- Identification of algorithms implemented in attention layers, distributed across attention headsanswered_byVPD successfully recovered interpretable attention algorithms (previous-token behavior, syntax-boundary routing) in weight space without requiring manual decomposition across heads.
- Specific discovered subcomponent that activates on punctuation like ' :', ' ;', ' =', ':-' and predicts the rest of emoticons/emojis.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Central thesis.
- Opening sentence setting the stage for the importance of interpretability.
- Sparse interpreter models may represent and decompose computational mechanisms differentlyclaim0.804Hinted by differences.
- A better basis (e.g., factoring tokens into languages or parts of speech) may yield a sparse interpretable modelhypothesis0.803Same model can be dense in one basis and sparse in another.
- Antonym example.
- Articulates why a one-layer transformer with MLP is the appropriate starting target for mechanistic interpretability
- Explicit hedge.
- Language models implement algorithms humans have tried and failed to write by hand for decadesclaim0.786Opening interpretive claim about the remarkable nature of language models.