method
active
method:qk-attributionsQK Attributions
Core method decomposing attention scores as bilinear sums of query-key feature pairs.
Neighborhood — ranked by edge-count
Papers (2)
paper
- Tracing Attention Computation Through Feature Interactionsintroducesmentions
Methods (2)
method
- QK and OV attributionsrelated_toWrites attention scores/head output as sums of feature interactions; core HeadVis capability.
- Attribution GraphsextendsGradient-based technique using SAE features to estimate causal effects on completions; used to corroborate NLA findings.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Future work.
- Central claim.
- Quadratic scaling.
- Measure causal importance of upstream variables.
- The task of attributing model behaviors to specific training datapoints.
- QK attribution from Ivory to 'a' has top pair Ivory Coast query feature and Zambian key featurefinding0.782Explains attention edge.
- Correlating attribution vectors (feature activation × logit weight of next token) across model pairs to measure functional universality
- Preliminary analyses suggest QK attribution matrices per head and position pair are approximately low-rankfinding0.771Motivates distillation.