paper:arxiv-2602-15029Symmetry in language statistics shapes the geometry of model representations
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- A geometric relation of the error introduced by sampling a language model's output distribution to its internal stateAlbert F. Modenbach2026≈ 77%
- The Wittgensteinian Representation Hypothesis: Is Language the Attractor of Multimodal Convergence?Run Shao, Dongyue Wu, Jiajie Teng, Chao Tao, Jingdong Chen, Haifeng Li Zhaoyang Zhang2026≈ 75%
- Revisiting Anisotropy in Language Transformers: The Geometry of Learning DynamicsFanny Jourdan, Antonin Poch\'e, C\'eline Hudelot Raphael Bernas2026≈ 75%
- Singular Vectors of Attention Heads Align with FeaturesCarson Loughridge, Mark Crovella Gabriel Franco2026≈ 74%
- Entangled in Representations: Mechanistic Investigation of Cultural Biases in Large Language ModelsSeogyeong Jeong, Siddhesh Pawar, Jisu Shin, Jiho Jin, Junho Myung, Alice Oh, Isabelle Augenstein Haeun Yu2026≈ 74%
- Unsupervised Concept Vector Extraction for Bias Control in LLMsYangfeng Ji, David Evans Hannah Cyberey2025≈ 74%
- Perceptions of Linguistic Uncertainty by Language Models and HumansMarkelle Kelly, Mark Steyvers, Sameer Singh, Padhraic Smyth Catarina G Belem2024≈ 74%
- Tradeoffs Between Alignment and Helpfulness in Language Models with Steering MethodsNoam Wies, Dorin Shteyman, Binyamin Rothberg, Yoav Levine, and Amnon Shashua Yotam Wolf2025≈ 74%
- The Confidence Manifold: Geometric Structure of Correctness Representations in Language ModelsZekun Wu, Kleyton Da Costa, Adriano Koshiyama Seonglae Cho2026≈ 74%
- Counterfactually Probing Language Identity in Multilingual ModelsVenkata S Govindarajan, Kyle Mahowald Anirudh Srinivasan2023≈ 74%
- Across the Levels of Analysis: Explaining Predictive Processing in Humans Requires More Than Machine-Estimated ProbabilitiesSathvik Nair and Colin Phillips2026≈ 73%
- ≈ 73%
- Dissecting Bias in LLMs: A Mechanistic Interpretability PerspectiveZubair Bashir, Procheta Sen Bhavik Chandna2025≈ 73%
- The Same But Different: Structural Similarities and Differences in Multilingual Language ModelingQinan Yu, Matianyu Zang, Carsten Eickhoff, Ellie Pavlick Ruochen Zhang2024≈ 73%
- Social Meaning in Large Language Models: Structure, Magnitude, and Pragmatic PromptingRoland M\"uhlenbernd2026≈ 73%
- ≈ 72%
- The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetsin corpus2023≈ 72%
- Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behaviorin corpus2026≈ 70%
- ≈ 70%
- Model Alignment Searchin corpus2025≈ 69%
- ≈ 69%
- The Platonic Representation Hypothesisin corpus2024≈ 69%
- ≈ 68%
- ≈ 68%
- Frieze Patterns of the Alhambrain corpus2007≈ 68%
- ≈ 68%
- ≈ 68%
Similar preprints — Semantic Scholar
Cited by (4)
- Do Sparse Autoencoders Capture Concept Manifolds?
Sparse autoencoders trained on Llama3.1-8B (layer 19, 500M Pile tokens, 25 SAEs across Standard, JumpReLU, TopK, BatchTopK and Matryoshka architectures at VE > 0.85) do not capture continuous concept
- Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds
Lifting the featurizer's atom from a direction to a block of directions recovers concept manifolds that SAEs shatter, and block sparsity is the MAP-matched prior for a representation that is a sparse
- Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts
Llama-3.1-8B solves cyclic arithmetic (e.g., "what month is six months after August?") not by performing modular addition in the period of the cyclic concept (12 for months, 7 for days of the week) as
- Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior
Manifold steering — intervening on model activations along paths constrained to lie on a learned activation manifold M_h rather than along Euclidean linear directions — produces behavioral trajectorie