How can mechanistic interpretability methods automatically identify attention computations that span multiple attention heads?

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Attention algorithms are usually distributed across attention headsclaim0.806
  Claim supported by VPD's recovery of cross-head attention subcomponents, noted in footnote.
• Attention computations distribute across heads via parameter subcomponents with interpretable rolesfinding0.804
  Mechanistic discovery about how attention mechanisms decompose into interpretable parameter components.
• The field of interpretability has focused mainly on understanding model activations, not the computations themselvesclaim0.793
  Motivation for VPD's parameter-focused approach.
• We hypothesize that interpretability methods could provide further evidence about indicators in particular systems or serve as the basis for distinct tests for consciousness.hypothesis0.778
  Forward-looking suggestion for how inner interpretability could extend the indicator method
• Attention probing can serve as an efficient tool for detecting performative reasoning and enabling adaptive computation in reasoning modelshypothesis0.777
  Forward-looking hypothesis positioned as a conclusion and future direction of the paper
• attention computationconcept0.776
  Process using Q, K, V to compute a heat map over K and weighted sum of V.
• Each attention head has two largely independent computations: a QK circuit computing the attention pattern and an OV circuit computing the effect if attended toclaim0.775
  Key decomposition enabling separate analysis of where attention goes and what it does
• Attention heads can be understood as independent operations each adding their output to the residual stream, equivalent to the concatenate-and-multiply formulationclaim0.770
  Mathematical equivalence enabling independent analysis of each attention head