claim
active
claim:lms-retrieve-bound-entities-via-a-mixture-of-positional-lexical-and-reflexive-mechanisms-rather-than-positional-aloneLMs retrieve bound entities via a mixture of positional, lexical and reflexive mechanisms rather than positional alone
Central thesis.
Source paper
extracted_from(2025) · Yoav Gur-Arieh · Mor Geva · Atticus Geiger
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (12)
finding
- Patching an unresolvable pointer at layer ℓ yields no counterfactual answer, but at ℓ+1 yields itsupportsValidates reflexive pointer.
- Pattern replicates across nine models (2–72B) and ten tasksassociated_withGeneralization.
- Ablation depends on t_entity.
- Ablation.
- Layer localization.
- Blocking attention to query token in layers 11–16 drops accuracy from 98% to 37%; with next token 8%supportsAttention knockout.
- 95% agreement.
- Interaction patterns in Figure 3.
- Upper bound.
- Fig. 19.
- Baseline comparison.
- Modulation by target position.
Claims (3)
claim
- Challenges prevailing view.
- Lexical and reflexive mechanisms provide sharper signals refining the diffuse positional signalextendsTakeaway.
- Architectural explanation.
Questions (1)
question
- Confounder resolved in §3.4.
Quotes (1)
quote
- Framing of challenged view.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Cross-model consistency.
- §D.3.
- Out-of-context reasoning work directly related to synthetic document fine-tuning experiments
- When lexical/reflexive targets absent from original, model falls back to noisy positional mechanismfinding0.769Fig. 28–29.
- ΦID analysis of attention-head activations across model families showing synergy concentrated in middle layers.
- Understanding how LMs learn linguistic behaviours may offer insights into fundamental properties of languagehypothesis0.763Forward-looking hypothesis linking LM mechanism analysis to linguistic theory
- Sharma et al. result supporting cross-modal alignment: language-only models implicitly encode visual structure
- discussion of potential confounds