paper:gao-scaling-and-evaluating-sparse-autoencode-2025Scaling and evaluating sparse autoencoders
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- ≈ 77%
- A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language ModelsXuansheng Wu, Haiyan Zhao, Daking Rai, Ziyu Yao, Ninghao Liu, Mengnan Du Dong Shu2025≈ 77%
- Sparse Autoencoder Features for Classifications and TransferabilityShan Chen, Kuleen Sasse, Hugo Aerts, Thomas Hartvigsen, Danielle S. Bitterman Jack Gallifant2026≈ 76%
- Evaluating and Designing Sparse Autoencoders by Approximating Quasi-OrthogonalityAdam Davies, Marc E. Canby and Julia Hockenmaier Sewoong Lee2025≈ 76%
- SOSAE: Self-Organizing Sparse AutoEncoderZi Pong Lim, Yushi Cao, Yupeng Cheng, Yon Shin Teo, Shang-Wei Lin Sarthak Ketanbhai Modi2025≈ 75%
- ≈ 75%
- Train One Sparse Autoencoder Across Multiple Sparsity Budgets to Preserve Interpretability and AccuracyYaroslav Aksenov, Daniil Laptev, Vadim Kurochkin, Gleb Gerasimov, Nikita Koryagin, Daniil Gavrilov Nikita Balagansky2025≈ 74%
- Stable and Steerable Sparse Autoencoders with Weight RegularizationOliver M. Crook Piotr Jedryszek2026≈ 74%
- Supervised sparse auto-encoders for interpretable and compositional representationsHugo Wallner, Yoonsoo Nam, Haixuan Xavier Tao Ouns El Harzli2026≈ 74%
- ≈ 74%
- ≈ 74%
- SplInterp: Improving our Understanding and Training of Sparse AutoencodersJavier Ideami, Benjamin Macdowall Rynne, Keith Duggar, Randall Balestriero Jeremy Budd2025≈ 74%
- Group Equivariance Meets Mechanistic Interpretability: Equivariant Sparse AutoencodersAna Lucic Ege Erdogan2025≈ 73%
- MetaSAEs: Joint Training with a Decomposability Penalty Produces More Atomic Sparse Autoencoder LatentsMatthew Levinson2026≈ 73%
- Sparse autoencoders reveal selective remapping of visual concepts during adaptationJinho Choi, Jaegul Choo, Steffen Schneider Hyesu Lim2025≈ 73%
- ≈ 73%
- ≈ 72%
- SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilityin corpus2025≈ 71%
- ≈ 70%
- Interpreting Language Model Parametersin corpus2026≈ 70%
- Do Sparse Autoencoders Capture Concept Manifolds?in corpus2026≈ 69%
- ≈ 69%
- ≈ 69%
- Mechanistic Knobs in LLMs: Retrieving and Steering High-Order Semantic Features via Sparse Autoencodersin corpus2026≈ 68%
- ≈ 67%
- Open Problems in Mechanistic Interpretabilityin corpus2025≈ 67%
- ≈ 67%
- ≈ 66%
- ≈ 66%
Similar preprints — Semantic Scholar
Cited by (7)
- Understanding sparse autoencoder scaling in the presence of feature manifolds
Sparse autoencoder scaling laws can break down when activations contain feature manifolds: if per-feature loss falls slowly with latent count, an SAE spends its latents tiling common manifolds instead
- Do Sparse Autoencoders Capture Concept Manifolds?
Sparse autoencoders trained on Llama3.1-8B (layer 19, 500M Pile tokens, 25 SAEs across Standard, JumpReLU, TopK, BatchTopK and Matryoshka architectures at VE > 0.85) do not capture continuous concept
- Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds
Lifting the featurizer's atom from a direction to a block of directions recovers concept manifolds that SAEs shatter, and block sparsity is the MAP-matched prior for a representation that is a sparse
- Turn-Averaged SAEs for Feature Discovery and Long-Context Attribution
Training a sparse autoencoder on the mean residual-stream activation of a conversation turn, rather than on individual tokens, yields features that describe a turn's high-level properties more complet
- SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
Gains on the sparsity-fidelity frontier do not predict how well a sparse autoencoder performs on downstream tasks, and SAEBench demonstrates this by scoring over 200 SAEs on eight metrics. Matryoshka
- Persona Features Control Emergent Misalignment
Fine-tuning GPT-4o on narrowly incorrect datasets—spanning nine domains including health, legal, and automotive advice—generalizes into broadly malicious behavior across unrelated prompts, confirming
- Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior
Manifold steering — intervening on model activations along paths constrained to lie on a learned activation manifold M_h rather than along Euclidean linear directions — produces behavioral trajectorie