paper:arxiv-2301-05217Progress measures for grokking via mechanistic interpretability
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- The Geometric Inductive Bias of Grokking: Bypassing Phase Transitions via Architectural TopologyAlper Y{\i}ld{\i}r{\i}m2026≈ 73%
- ≈ 73%
- ≈ 72%
- Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language ModelsZhihao Zhang, Mingyang Wang, Zunhai Su, Yiwei Wang, Qianli Wang, Shuzhou Yuan, Ercong Nie, Xufeng Duan, Feijiang Han, Qibo Xue, Zeping Yu, Chenming Shang, Xiao Liang, Jing Xiong, Hui Shen, Chaofan Tao, Zhengwu Liu, Senjie Jin, Zhiheng Xi, Dongdong Zhang, Sophia Ananiadou, Tao Gui, Ruobing Xie, Hayden Kwok-Hay So, Hinrich Sch\"utze, Xuanjing Huang, Qi Zhang, Ngai Wong Hengyuan Zhang2026≈ 72%
- Beyond Transcription: Mechanistic Interpretability in ASRYael Segal-Feldman, Hilit Segev, Aviv Shamsian, Asaf Buchnick, Gill Hetz, Ethan Fetaya, Joseph Keshet, Aviv Navon Neta Glazer2025≈ 71%
- Position: Mechanistic Interpretability Must Disclose Identification Assumptions for Causal ClaimsFengming Liu Zezheng Lin2026≈ 71%
- Mechanistic interpretability of large language models with applications to the financial services industryKhashayar Filom, and Arjun Ravi Kannan Ashkan Golgoon2024≈ 71%
- A Practical Review of Mechanistic Interpretability for Transformer-Based Language ModelsYilun Zhou, Shi Feng, Abulhair Saparov, Ziyu Yao Daking Rai2025≈ 71%
- Mechanistic Unveiling of Transformer Circuits: Self-Influence as a Key to Model ReasoningLijie Hu, Di Wang Lin Zhang2025≈ 71%
- Open Problems in Mechanistic Interpretabilityin corpus2025≈ 71%
- Mechanistic Interpretability as Statistical Estimation: A Variance AnalysisFran\c{c}ois Portet, Maxime Peyrard Maxime M\'eloux2026≈ 71%
- Interpretability and Generalization Bounds for Learning Spatial PhysicsTheo Gutman-Solo, Shuai Jiang Alejandro Francisco Queiruga2026≈ 71%
- Mechanistic Interpretability Needs PhilosophyNinell Oldenburg, Ruchira Dhar, Joshua Hatherley, Constanza Fierro, Nina Rajcic, Sandrine R. Schiller, Filippos Stamatiou, Anders S{\o}gaard Iwan Williams2025≈ 71%
- Validating Mechanistic Interpretations: An Axiomatic ApproachRavi Mangal, Zifan Wang, Saranya Vijayakumar, Corina S. Pasareanu, Somesh Jha Nils Palumbo2025≈ 71%
- Interpretability without actionability: mechanistic methods cannot correct language model errors despite near-perfect internal representationsSadiq Y. Patel, Parth Sheth, Bhairavi Muralidharan, Namrata Elamaran, Aakriti Kinra, John Morgan, Rajaie Batniji Sanjay Basu2026≈ 70%
- ≈ 70%
- ≈ 69%
- ≈ 68%
- ≈ 68%
- ≈ 68%
- Mechanistic Knobs in LLMs: Retrieving and Steering High-Order Semantic Features via Sparse Autoencodersin corpus2026≈ 68%
- ≈ 67%
- ≈ 67%
- ≈ 67%
- Quantitative Introspection in Language Models: Tracking Emotive States Across Conversationin corpus2026≈ 67%
- Evaluating Language Model Character Traitsin corpus2024≈ 67%
- ≈ 67%
- ≈ 67%
- The Platonic Representation Hypothesisin corpus2024≈ 67%
Similar preprints — Semantic Scholar
Cited by (5)
- Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention
Larger models learn tasks smaller models never learn, and the cause is data-induced competition for neurons, not expressivity: small models spend capacity on frequent or simple tasks, and their update
- Forking Fast: Efficiently Estimating Uncertainty Dynamics in Text Generation
Resampling noise in reasoning-chain uncertainty dynamics is almost exactly multinomial, so most of the cost of Forking Paths Analysis re-measures structure that neighboring tokens already contain. On
- When Models Manipulate Manifolds: The Geometry of a Counting Task
Claude 3.5 Haiku's linebreaking in fixed-width text is implemented by manipulating low-dimensional, rippled 1-D feature manifolds: character count, line width, characters remaining and next-token leng
- From Directions to Cones: Exploring Multidimensional Representations of Propositional Facts in LLMs
Propositional truth in LLMs is not encoded as a single linear direction but as a multi-dimensional subspace that can be characterized by concept cones—sets of all nonnegative linear combinations of or
- Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts
Llama-3.1-8B solves cyclic arithmetic (e.g., "what month is six months after August?") not by performing modular addition in the period of the cyclic concept (12 for months, 7 for days of the week) as