claim
active
claim:weakening-lexical-relative-to-noisy-positional-mechanism-may-mechanistically-explain-lost-in-the-middleWeakening lexical relative to noisy positional mechanism may mechanistically explain lost-in-the-middle
Interpretation of padding results.
Source paper
extracted_from(2025) · Yoav Gur-Arieh · Mor Geva · Atticus Geiger
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (2)
finding
- At 10,000 tokens positional information is nearly nonexistent for first half of groups except firstsupportsPadding result.
- Long-context generalization.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- LM weakness on middle context positions, echoed here.
- Challenges prevailing view.
- When lexical/reflexive targets absent from original, model falls back to noisy positional mechanismfinding0.765Fig. 28–29.
- Learned parameters.
- Lexical and reflexive mechanisms provide sharper signals refining the diffuse positional signalclaim0.734Takeaway.
- Splitting the loss into predictive-only and novel-residual terms breaks the competition dynamic.hypothesis0.729Proposed training fix inspired by Kalman filters.
- Central thesis.
- §D.3.