claim
active
claim:all-induction-heads-fall-in-an-extreme-corner-of-high-ov-eigenvalue-positivity-and-high-qk-eigenvalue-positivity-confirming-the-mechanistic-theoryAll induction heads fall in an extreme corner of high OV eigenvalue positivity and high QK eigenvalue positivity, confirming the mechanistic theory
Quantitative verification that the copying and matching structure predicted by the mechanistic theory is present in all observed induction heads
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Quantitative verification of the mechanistic theory; both circuits required for the induction algorithm show the predicted copying/matching structure
- Basis-independent corroboration.
- Forward-looking claim connecting toy model findings to large-scale language models
- The mechanistic explanation of how induction heads are implemented in two-layer models
- Early-layer induction heads do literal copying while mid-layer heads do fuzzier inductionfinding0.765Layer-dependent induction on Haiku 3.5.
- Central empirical claim of the paper; induction heads are shown to be the mechanism for powerful in-context learning
- Structural finding about which attention heads control reflection behavior
- Replication of prosocial paradox on reasoning model