finding
active
finding:blocking-query-to-all-entities-attention-leaves-lexical-accuracy-at-90Blocking query-to-all-entities attention leaves lexical accuracy at 90%
Lexical signal local.
Source paper
extracted_from(2025) · Yoav Gur-Arieh · Mor Geva · Atticus Geiger
Neighborhood — ranked by edge-count
Papers (1)
paper
Hypotheses (1)
hypothesis
- Tested with knockout.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Reflexive signal retrieved from entity tokens.
- Blocking attention to query token in layers 11–16 drops accuracy from 98% to 37%; with next token 8%finding0.803Attention knockout.
- Reward probes attend to entity and relation tokens; Failed Fix shows higher attention on punctuationfinding0.750Attention map observation.
- Interesting special case of copying behavior related to tokenization artifacts; primitive precursor to induction heads
- QK attribution for MC heads.
- §D.3.
- Dismissal of earlier criteria as too narrow.
- Design choice validated empirically.