paper:bussmann-batchtopk-sparse-autoencoders-2024Batchtopk sparse autoencoders
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- SplInterp: Improving our Understanding and Training of Sparse AutoencodersJavier Ideami, Benjamin Macdowall Rynne, Keith Duggar, Randall Balestriero Jeremy Budd2025≈ 71%
- Train One Sparse Autoencoder Across Multiple Sparsity Budgets to Preserve Interpretability and AccuracyYaroslav Aksenov, Daniil Laptev, Vadim Kurochkin, Gleb Gerasimov, Nikita Koryagin, Daniil Gavrilov Nikita Balagansky2025≈ 71%
- Evaluating and Designing Sparse Autoencoders by Approximating Quasi-OrthogonalityAdam Davies, Marc E. Canby and Julia Hockenmaier Sewoong Lee2025≈ 70%
- SparseRM: A Lightweight Preference Modeling with Sparse AutoencoderJiahao Li, Zheren Fu, Yi Tu, Jiajun Li, Zhendong Mao, Yongdong Zhang Dengcan Liu2025≈ 70%
- Supervised sparse auto-encoders for interpretable and compositional representationsHugo Wallner, Yoonsoo Nam, Haixuan Xavier Tao Ouns El Harzli2026≈ 70%
- SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilityin corpus2025≈ 70%
- Improving Sparse Autoencoder with Dynamic AttentionJinsen Zhang, Dawei Su, Hui Huang Dongsheng Wang2026≈ 70%
- ≈ 70%
- SOSAE: Self-Organizing Sparse AutoEncoderZi Pong Lim, Yushi Cao, Yupeng Cheng, Yon Shin Teo, Shang-Wei Lin Sarthak Ketanbhai Modi2025≈ 69%
- Sparse Autoencoder Features for Classifications and TransferabilityShan Chen, Kuleen Sasse, Hugo Aerts, Thomas Hartvigsen, Danielle S. Bitterman Jack Gallifant2026≈ 69%
- SoftSAE: Dynamic Top-K Selection for Adaptive Sparse AutoencodersMarcin Mazur, Jacek Tabor, Przemys{\l}aw Spurek Jakub St\k{e}pie\'n2026≈ 69%
- Interpreting Attention Layer Outputs with Sparse AutoencodersRobert Krzyzanowski, Joseph Isaac Bloom, Arthur Conmy, Neel Nanda Connor Kissane2024≈ 69%
- ≈ 69%
- Sparse autoencoders reveal selective remapping of visual concepts during adaptationJinho Choi, Jaegul Choo, Steffen Schneider Hyesu Lim2025≈ 69%
- MetaSAEs: Joint Training with a Decomposability Penalty Produces More Atomic Sparse Autoencoder LatentsMatthew Levinson2026≈ 69%
- Sparse Autoencoder Decomposition of Clinical Sequence Model Representations: Feature Complexity, Task Specialisation, and Mortality PredictionFeng Dong, Andreas Karwath Chris Sainsbury2026≈ 69%
- Do Sparse Autoencoders Capture Concept Manifolds?in corpus2026≈ 66%
- ≈ 66%
- ≈ 65%
- ≈ 64%
- ≈ 64%
- Open Problems in Mechanistic Interpretabilityin corpus2025≈ 64%
- ≈ 63%
- Mechanistic Knobs in LLMs: Retrieving and Steering High-Order Semantic Features via Sparse Autoencodersin corpus2026≈ 63%
- Interpreting Language Model Parametersin corpus2026≈ 63%
- ≈ 63%
- ≈ 63%
- ≈ 63%
- ≈ 62%
Similar preprints — Semantic Scholar
Cited by (5)
- Understanding sparse autoencoder scaling in the presence of feature manifolds
Sparse autoencoder scaling laws can break down when activations contain feature manifolds: if per-feature loss falls slowly with latent count, an SAE spends its latents tiling common manifolds instead
- Do Sparse Autoencoders Capture Concept Manifolds?
Sparse autoencoders trained on Llama3.1-8B (layer 19, 500M Pile tokens, 25 SAEs across Standard, JumpReLU, TopK, BatchTopK and Matryoshka architectures at VE > 0.85) do not capture continuous concept
- Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds
Lifting the featurizer's atom from a direction to a block of directions recovers concept manifolds that SAEs shatter, and block sparsity is the MAP-matched prior for a representation that is a sparse
- Turn-Averaged SAEs for Feature Discovery and Long-Context Attribution
Training a sparse autoencoder on the mean residual-stream activation of a conversation turn, rather than on individual tokens, yields features that describe a turn's high-level properties more complet
- Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior
Manifold steering — intervening on model activations along paths constrained to lie on a learned activation manifold M_h rather than along Euclidean linear directions — produces behavioral trajectorie