finding
active
finding:w-qk-feature-interactions-and-eigenvalues-skew-positive-for-concordance-heads-and-negative-for-discordance-headsW_QK feature interactions and eigenvalues skew positive for concordance heads and negative for discordance heads
Basis-independent corroboration.
Source paper
extracted_fromNeighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Preliminary conclusion.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Head-level causal evidence.
- Asymmetry suggesting more things must go right.
- Concordance heads (QK circuits) could serve as the consistency-checking circuit for distinguishing intended vs. unintended outputshypothesis0.789Speculated mechanism for prefill detection.
- Heads that detect concordance or discordance of a stated answer with the posed computation.
- Quantitative verification that the copying and matching structure predicted by the mechanistic theory is present in all observed induction heads
- Reframing observation: the canonical K/Q/V decomposition is computationally convenient but not the most interpretable representation
- Generalization to arithmetic.
- Interpretation of steering asymmetry.