method
active
method:qk-and-ov-attributionsQK and OV attributions
Writes attention scores/head output as sums of feature interactions; core HeadVis capability.
Neighborhood — ranked by edge-count
Papers (1)
paper
Methods (2)
method
- QK Attributionsrelated_toCore method decomposing attention scores as bilinear sums of query-key feature pairs.
- Source of the features used in attributions.
Artifacts (1)
artifact
- HeadVisimplementsInteractive tool for visualizing attention heads across the full data distribution; frontend open-sourced.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Central claim.
- Future work.
- Quadratic scaling.
- QK attribution from Ivory to 'a' has top pair Ivory Coast query feature and Zambian key featurefinding0.780Explains attention edge.
- Preliminary analyses suggest QK attribution matrices per head and position pair are approximately low-rankfinding0.761Motivates distillation.
- In the Aunt Mary Sue prompt, QK attributions matrix has rank at least 2 with no strong cross termsfinding0.756Parallel independent QK interactions.
- Stronger version: all cognition attributions rely on observable behavior.
- Correlating attribution vectors (feature activation × logit weight of next token) across model pairs to measure functional universality