paper
referenced-only
2025
paper:arxiv-2504-12459On linear representations and pretraining data frequency in language models
ByJack Merullo·Noah A. Smith·Sarah Wiegreffe·Yanai Elazar
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- What do Language Models Learn and When? The Implicit Curriculum HypothesisKaiser Sun, Millicent Li, Isabelle Lee, Lindia Tjuatja, Jen-tse Huang, Graham Neubig Emmy Liu2026≈ 78%
- Across the Levels of Analysis: Explaining Predictive Processing in Humans Requires More Than Machine-Estimated ProbabilitiesSathvik Nair and Colin Phillips2026≈ 78%
- A Survey of Large Language ModelsKun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie and Ji-Rong Wen Wayne Xin Zhao2026≈ 77%
- SocioProbe: What, When, and Where Language Models Learn about SociodemographicsFederico Bianchi, Samuel Bowman, and Dirk Hovy Anne Lauscher2022≈ 76%
- Pretraining Exposure Explains Popularity Judgments in Large Language ModelsBhawna Piryani, Adam Jatowt Jamshid Mozafari2026≈ 76%
- Evaluating Neural Language Models as Cognitive Models of Language AcquisitionAnnika Lea Heuser, Charles Yang, Jordan Kodner H\'ector Javier V\'azquez Mart\'inez2026≈ 75%
- When Do You Need Billions of Words of Pretraining Data?Alex Warstadt, Haau-Sing Li, and Samuel R. Bowman Yian Zhang2020≈ 75%
- Improving Language Models Trained on Translated Data with Continual Pre-Training and Dictionary Learning AnalysisMD Rizwan Parvez, Majd Hawasly Sabri Boughorbel2024≈ 75%
- Probing Task-Oriented Dialogue Representation from Language ModelsChien-Sheng Wu and Caiming Xiong2020≈ 75%
- Reanalyzing L2 Preposition Learning with Bayesian Mixed Effects and a Pretrained Language ModelJakob Prange and Man Ho Ivy Wong2026≈ 75%
- Patches of Nonlinearity: Instruction Vectors in Large Language ModelsJonas Rohweder, Subhabrata Dutta, Iryna Gurevych Irina Bigoulaeva2026≈ 75%
- Identifying and Mitigating the Influence of the Prior Distribution in Large Language ModelsVeniamin Veselovsky, R. Thomas McCoy, Thomas L. Griffiths Liyi Zhang2025≈ 75%
- ≈ 74%
- From Parameters to Performance: A Data-Driven Study on LLM Structure and DevelopmentZuchao Li, Luohe Shi, Bo Du, Hai Zhao, Yun Li, Qianren Wang Suqing Wang2025≈ 74%
- Scaling Laws for Downstream Task Performance of Large Language ModelsNatalia Ponomareva, Hussein Hazimeh, Dimitris Paparas, Sergei Vassilvitskii, Sanmi Koyejo Berivan Isik2026≈ 74%
- Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retentionin corpus2026≈ 72%
- ≈ 72%
- ≈ 72%
- ≈ 71%
- ≈ 71%
- ≈ 71%
- The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetsin corpus2023≈ 71%
- ≈ 70%
- ≈ 70%
- Evaluating Language Model Character Traitsin corpus2024≈ 70%
- Probe-Based Data Attribution: Surfacing and Mitigating Undesirable Behaviors in LLM Post-Trainingin corpus2026≈ 69%
- ≈ 69%
- The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?in corpus2025≈ 69%
- Interpreting Language Model Parametersin corpus2026≈ 69%
- ≈ 69%
Similar preprints — Semantic Scholar
Cited by (2)
- From Memorization to Reasoning in the Spectrum of Loss Curvature
- Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior
Manifold steering — intervening on model activations along paths constrained to lie on a learned activation manifold M_h rather than along Euclidean linear directions — produces behavioral trajectorie