paper:arxiv-2505-18235The origins of representation manifolds in large language models
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- A Survey of Large Language ModelsKun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie and Ji-Rong Wen Wayne Xin Zhao2026≈ 80%
- Towards Uncovering How Large Language Model Works: An Explainability PerspectiveFan Yang, Bo Shen, Himabindu Lakkaraju, Mengnan Du Haiyan Zhao2024≈ 77%
- Across the Levels of Analysis: Explaining Predictive Processing in Humans Requires More Than Machine-Estimated ProbabilitiesSathvik Nair and Colin Phillips2026≈ 76%
- Advancing the Scientific Method with Large Language Models: From Hypothesis to DiscoverySumeer A. Khan, Adnan Mahmud, Huck Yang, Alexander Lavin, Michael Levin, Jeremy Frey, Jared Dunnmon, James Evans, Alan Bundy, Saso Dzeroski, Jesper Tegner, Hector Zenil Yanbo Zhang2025≈ 76%
- What do Language Models Learn and When? The Implicit Curriculum HypothesisKaiser Sun, Millicent Li, Isabelle Lee, Lindia Tjuatja, Jen-tse Huang, Graham Neubig Emmy Liu2026≈ 76%
- Exploring Geographic Relative Space in Large Language Models through Activation PatchingRahul Baiju, Stefano Mizzaro, Kevin Roitero Stef De Sabbata2026≈ 76%
- The Birth of Knowledge: Emergent Features across Time, Space, and Scale in Large Language ModelsMicah Adler, Nir Shavit Shashata Sawmya2025≈ 75%
- ≈ 75%
- ≈ 75%
- Reasoning emerges from constrained inference manifolds in large language modelsFei Luo, Linfeng Zhang, Chuangxin Zhao, Mingxuan Wang, Yinan Wu, Zhe Qian, Yang Lu, Long Chen, Zhao Cao, Xiaoshuai Hao, Ji-Rong Wen, Jungong Han Yanbiao Ma2026≈ 75%
- ≈ 75%
- Maistros: A Greek Large Language Model Adapted Through Knowledge Distillation From Large Reasoning ModelsCharalampos Mastrokostas, Nikos Karacapilidis Nikolaos Giarelis2026≈ 74%
- Taxonomy, Opportunities, and Challenges of Representation Engineering for Large Language ModelsSahar Abdelnabi, Daniel Tan, David Krueger, Mario Fritz Jan Wehner2025≈ 74%
- ≈ 74%
- Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-ThoughtAbulhair Saparov and He He2023≈ 74%
- ≈ 74%
- ≈ 72%
- The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetsin corpus2023≈ 72%
- ≈ 71%
- Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behaviorin corpus2026≈ 71%
- ≈ 71%
- Do Sparse Autoencoders Capture Concept Manifolds?in corpus2026≈ 70%
- The World Inside Neural Networksin corpus2026≈ 70%
- ≈ 70%
- ≈ 70%
- ≈ 69%
- Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retentionin corpus2026≈ 69%
- ≈ 69%
- ≈ 69%
Similar preprints — Semantic Scholar
Cited by (7)
- Understanding and Steering Llama 3 with Sparse Autoencoders - Goodfire
A sparse autoencoder trained on layer-19 residual-stream activations of Llama-3-8B-Instruct, using LMSYS-Chat-1M, yields features that can be labeled automatically, ranked by causal relevance, and use
- Understanding sparse autoencoder scaling in the presence of feature manifolds
Sparse autoencoder scaling laws can break down when activations contain feature manifolds: if per-feature loss falls slowly with latent count, an SAE spends its latents tiling common manifolds instead
- Do Sparse Autoencoders Capture Concept Manifolds?
Sparse autoencoders trained on Llama3.1-8B (layer 19, 500M Pile tokens, 25 SAEs across Standard, JumpReLU, TopK, BatchTopK and Matryoshka architectures at VE > 0.85) do not capture continuous concept
- Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds
Lifting the featurizer's atom from a direction to a block of directions recovers concept manifolds that SAEs shatter, and block sparsity is the MAP-matched prior for a representation that is a sparse
- When Models Manipulate Manifolds: The Geometry of a Counting Task
Claude 3.5 Haiku's linebreaking in fixed-width text is implemented by manipulating low-dimensional, rippled 1-D feature manifolds: character count, line width, characters remaining and next-token leng
- Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior
Manifold steering — intervening on model activations along paths constrained to lie on a learned activation manifold M_h rather than along Euclidean linear directions — produces behavioral trajectorie
- Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts
Llama-3.1-8B solves cyclic arithmetic (e.g., "what month is six months after August?") not by performing modular addition in the period of the cyclic concept (12 for months, 7 for days of the week) as