finding
active
finding:an-earlier-head-attends-from-a-to-chikumbuso-and-writes-the-zambian-features-read-by-the-suppression-headAn earlier head attends from 'a' to chikumbuso and writes the Zambian features read by the suppression head
K-composition example.
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Explains hard-to-read dataset patterns.
Concepts (1)
concept
- K-CompositionsupportsA form of attention head composition where W_K reads from a subspace affected by a previous head; central to how induction heads are implemented
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Replacing Ivory Coast with Zambia drops attention to 'a' from 0.87 to 0.08; Lusaka gives 0.10finding0.745Same-set suppression on real example.
- Custom sentences with odd one out.
- Characterizes what is on the far end of the Assistant Axis away from the Assistant
- Striking sparsity finding that enables surgical intervention
- Interesting special case of copying behavior related to tokenization artifacts; primitive precursor to induction heads
- Prediction for future decomposition.
- Mathematical equivalence enabling independent analysis of each attention head