finding
active
finding:qk-attribution-from-ivory-to-a-has-top-pair-ivory-coast-query-feature-and-zambian-key-featureQK attribution from Ivory to 'a' has top pair Ivory Coast query feature and Zambian key feature
Explains attention edge.
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Explains hard-to-read dataset patterns.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Replacing Ivory Coast with Zambia drops attention to 'a' from 0.87 to 0.08; Lusaka gives 0.10finding0.788Same-set suppression on real example.
- Core method decomposing attention scores as bilinear sums of query-key feature pairs.
- Central claim.
- Writes attention scores/head output as sums of feature interactions; core HeadVis capability.
- Future work.
- Feature attribution (gradient-based) correlates 0.8 with ablation effects on the 'John' and 'Kobe' examples.finding0.756Validation of attribution as a fast proxy for causal importance.
- Preliminary analyses suggest QK attribution matrices per head and position pair are approximately low-rankfinding0.743Motivates distillation.
- An earlier head attends from 'a' to chikumbuso and writes the Zambian features read by the suppression headfinding0.734K-composition example.