Virtual attention heads (V-composition) may be much more important in larger and more complex transformers than in two-layer toy models

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• In small two-layer attention-only transformers, the only significant composition is K-composition between a single first-layer head and some second-layer headsclaim0.800
  Empirical observation from the specific two-layer model analyzed; no significant V- or Q-composition found
• Two-layer attention-only transformers implement much more complex algorithms via composition of attention heads, detectable directly from weightsclaim0.764
  Core claim for two-layer models; composition creates qualitatively more powerful in-context learning
• Most attention heads in one-layer models dedicate an enormous fraction of their capacity to copying behaviorclaim0.763
  Empirical observation from examining expanded OV/QK matrices; approximately 10 out of 12 heads show significant copying
• Naive interpretation of attention patterns can be both informative and fundamentally misleading when Q-, K-, or V-composition is presentclaim0.751
  Response to the 'attention as explanation' critique; the paper provides a typology of when attention is and isn't directly interpretable
• Do we 'fully understand' one-layer attention-only transformers?question0.751
  The paper explicitly asks and addresses this question, concluding the answer depends on what 'fully understand' means
• In the analyzed two-layer model, second-layer attention head terms dominate the loss reduction compared to first-layer terms and the direct pathfinding0.745
  Result from term importance analysis breaking down loss contribution by layer
• In the analyzed two-layer attention-only model, only K-composition is significant; V- and Q-composition are negligible by Frobenius norm measurefinding0.744
  Result from applying the Frobenius norm composition measurement to all attention head pairs in the two-layer model
• If models inhabit expanded attentional modes, they may be more aligned and less prone to psychosis and doom spirals.hypothesis0.741
  Speculative alignment implication drawn from the collapsed/expanded distinction.