finding
active
finding:with-up-to-10-000-filler-tokens-lexical-effect-declines-while-positional-and-mixed-effects-increase-on-gemma-2-2b-it-boxesWith up to 10,000 filler tokens, lexical effect declines while positional and mixed effects increase on gemma-2-2b-it boxes
Long-context generalization.
Source paper
extracted_from(2025) · Yoav Gur-Arieh · Mor Geva · Atticus Geiger
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Interpretation of padding results.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Layer localization.
- Fig. 10.
- Small Gemma model shows severe ASR degradation at higher cone dimensions
- Directly prompting Gemma-2-27B to be honest had no effect on deceptive response rate
- Interpretation of probe vs judge gap.
- Fig. 19.
- Positional mechanism accounts for only ~20% of behavior in middle entity groups for n=20 on gemma-2-2b-itfinding0.747Core result.
- Gemma-2-27B-it deceptive response rate reduced from 100% to 9.36% ± 7.09% after SOO fine-tuningfinding0.745Primary result showing SOO fine-tuning significantly reduces deception in Gemma-2-27B