finding

active

finding:attention-computations-distribute-across-heads-via-parameter-subcomponents-with-interpretable-roles

Attention computations distribute across heads via parameter subcomponents with interpretable roles

Mechanistic discovery about how attention mechanisms decompose into interpretable parameter components.

Source paper

extracted_from

Interpreting Language Model Parameters

(2026) · Bushnaq, Lucius · Braun, Dan · Clive-Griffin, Oliver · Bussmann, Bart +4

Neighborhood — ranked by edge-count

Communities (4)

community

Mechanistic interpretability & model evaluation
members_of
Spans attention head decomposition, benchmark awareness, and genomic pathogenicity prediction via neural models.
Mechanistic interpretability via parameter decomposition
members_of
Tracing information flow through weight matrices and attention heads using attribution graphs to identify causally important subcomponents in language models.
Causal parameter decomposition in neural networks
members_of
Isolating interpretable, role-specific model subcomponents through causal analysis and targeted edits to understand mechanistic function.
Distributed attention head decomposition
members_of
Mechanistic interpretability approach decomposing attention heads into query/key subcomponents with distinct algorithmic roles

Concepts (2)

concept

Parameter Decomposition (vs Activation)
supports
Attention heads
cites
Transformer attention heads that could be recruited to extract different kinds of information (text vs. thoughts).

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Attention algorithms are usually distributed across attention headsclaim0.834
Claim supported by VPD's recovery of cross-head attention subcomponents, noted in footnote.
Identification of algorithms implemented in attention layers, distributed across attention headsfinding0.809
VPD successfully recovered interpretable attention algorithms (previous-token behavior, syntax-boundary routing) in weight space without requiring manual decomposition across heads.
A pair of query and key subcomponents distributed across attention heads performs previous-token behaviorfinding0.806
VPD recovers an attention algorithm for attending to the previous token, distributed across multiple heads.
How can mechanistic interpretability methods automatically identify attention computations that span multiple attention heads?question0.804
Long-standing bottleneck in mechanistic interpretability that VPD addresses by working natively on attention weight matrices.
A pair of query and key subcomponents distributed across attention heads performs syntax-boundary routingfinding0.798
VPD recovers an attention algorithm for routing across syntactic boundaries, distributed across heads.
Attention heads can be understood as independent operations each adding their output to the residual stream, equivalent to the concatenate-and-multiply formulationclaim0.793
Mathematical equivalence enabling independent analysis of each attention head
Some attention heads partially specialize in copying for words that split into two tokens without a space prefix, attending from fragmented token to complete tokenfinding0.786
Interesting special case of copying behavior related to tokenization artifacts; primitive precursor to induction heads
are the resulting parameter subcomponents actually interpretable objects?question0.783
First question posed after applying VPD, investigating whether the subcomponents make sense.