paper:cunningham-sparse-autoencoders-find-highly-interpre-2023Sparse autoencoders find highly interpretable features in language models
Methods (1)
- Sparse Autoencoders (SAEs)Unsupervised baseline method for decomposing activations into sparse linear combinations; NLAs positioned as complementary approach.
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language ModelsXuansheng Wu, Haiyan Zhao, Daking Rai, Ziyu Yao, Ninghao Liu, Mengnan Du Dong Shu2025≈ 85%
- ≈ 83%
- Mechanistic Interpretability of ASR models using Sparse AutoencodersZachary Nicholas Houghton, Yu Zhou, and Vijay K. Gurbani Dan Pluth2026≈ 83%
- Sparse Autoencoder Features for Classifications and TransferabilityShan Chen, Kuleen Sasse, Hugo Aerts, Thomas Hartvigsen, Danielle S. Bitterman Jack Gallifant2026≈ 82%
- ≈ 82%
- ≈ 81%
- Evaluating and Designing Sparse Autoencoders by Approximating Quasi-OrthogonalityAdam Davies, Marc E. Canby and Julia Hockenmaier Sewoong Lee2025≈ 81%
- Interpreting Attention Layer Outputs with Sparse AutoencodersRobert Krzyzanowski, Joseph Isaac Bloom, Arthur Conmy, Neel Nanda Connor Kissane2024≈ 81%
- Group Equivariance Meets Mechanistic Interpretability: Equivariant Sparse AutoencodersAna Lucic Ege Erdogan2025≈ 80%
- Sparse Autoencoders Enable Scalable and Reliable Circuit Identification in Language ModelsThang Bui Charles O'Neill2024≈ 80%
- Features that Make a Difference: Leveraging Gradients for Improved Dictionary LearningJared Wilson, Max Forsey, Bryce Hepner, Thomas Vin Howe, David Wingate Jeffrey Olmo2025≈ 80%
- Analyze Feature Flow to Enhance Interpretation and Steering in Language ModelsNikita Balagansky, Yaroslav Aksenov, Daniil Gavrilov Daniil Laptev2025≈ 80%
- How Pruning Reshapes Features: Sparse Autoencoder Analysis of Weight-Pruned Language ModelsElies Segu\'i-Mas, Guillermina Tormo-Carb\'o Hector Borobia2026≈ 80%
- A Unified Theory of Sparse Dictionary Learning in Mechanistic Interpretability: Piecewise Biconvexity and Spurious MinimaHarshvardhan Saini, Zhaoqian Yao, Zheng Lin, Yizhen Liao, Jingyi Cui, Yisen Wang, Mengnan Du, Dianbo Liu Yiming Tang2026≈ 80%
- Beyond Input Activations: Identifying Influential Latents by Gradient Sparse AutoencodersXuansheng Wu, Haiyan Zhao, Mengnan Du, Ninghao Liu Dong Shu2025≈ 80%
- ≈ 77%
- Interpreting Language Model Parametersin corpus2026≈ 76%
- Mechanistic Knobs in LLMs: Retrieving and Steering High-Order Semantic Features via Sparse Autoencodersin corpus2026≈ 76%
- ≈ 76%
- Do Sparse Autoencoders Capture Concept Manifolds?in corpus2026≈ 75%
- ≈ 75%
- ≈ 75%
- ≈ 75%
- SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilityin corpus2025≈ 74%
- ≈ 74%
- Open Problems in Mechanistic Interpretabilityin corpus2025≈ 73%
- ≈ 73%
- Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retentionin corpus2026≈ 71%
- ≈ 71%
Similar preprints — Semantic Scholar
Cited by (18)
- Understanding and Steering Llama 3 with Sparse Autoencoders - Goodfire
A sparse autoencoder trained on layer-19 residual-stream activations of Llama-3-8B-Instruct, using LMSYS-Chat-1M, yields features that can be labeled automatically, ranked by causal relevance, and use
- Stochastic Parameter Decomposition
Rank-one subcomponents trained against stochastic ablation masks recover ground-truth mechanisms in toy networks that Attribution-based Parameter Decomposition (APD) could not decompose, while removin
- Adversarial Examples Are Not Bugs, They Are Superposition
Superposition causally contributes to adversarial vulnerability, and the link runs in both directions: in a toy ReLU autoencoder (n=100 features, m=20 hidden units), raising feature sparsity raises fe
- Understanding sparse autoencoder scaling in the presence of feature manifolds
Sparse autoencoder scaling laws can break down when activations contain feature manifolds: if per-feature loss falls slowly with latent count, an SAE spends its latents tiling common manifolds instead
- Do Sparse Autoencoders Capture Concept Manifolds?
Sparse autoencoders trained on Llama3.1-8B (layer 19, 500M Pile tokens, 25 SAEs across Standard, JumpReLU, TopK, BatchTopK and Matryoshka architectures at VE > 0.85) do not capture continuous concept
- Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds
Lifting the featurizer's atom from a direction to a block of directions recovers concept manifolds that SAEs shatter, and block sparsity is the MAP-matched prior for a representation that is a sparse
- Turn-Averaged SAEs for Feature Discovery and Long-Context Attribution
Training a sparse autoencoder on the mean residual-stream activation of a conversation turn, rather than on individual tokens, yields features that describe a turn's high-level properties more complet
- SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
Gains on the sparsity-fidelity frontier do not predict how well a sparse autoencoder performs on downstream tasks, and SAEBench demonstrates this by scoring over 200 SAEs on eight metrics. Matryoshka
- Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers
A single LLM trained to read injected residual-stream activations and answer arbitrary natural-language questions about them can audit fine-tuned models it never saw during training. Activation Oracle
- Would this change your answer? Evaluating Explanations of LLM Behavior In The Wild with Counterfactual Experiments
Activation-reading interpretability tools give no uplift for predicting counterfactual model behavior on naturally occurring behaviors, even though the same tool families help in narrow fine-tuned-qui
- Steering at the Source: Style Modulation Heads for Robust Persona Control
Residual-stream activation steering reliably degrades text coherency when steering vectors push models toward out-of-distribution behavior, and this collapse goes undetected by standard benchmarks: MM
- Persona Features Control Emergent Misalignment
Fine-tuning GPT-4o on narrowly incorrect datasets—spanning nine domains including health, legal, and automotive advice—generalizes into broadly malicious behavior across unrelated prompts, confirming
- From Directions to Cones: Exploring Multidimensional Representations of Propositional Facts in LLMs
Propositional truth in LLMs is not encoded as a single linear direction but as a multi-dimensional subspace that can be characterized by concept cones—sets of all nonnegative linear combinations of or
- The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?
Under arbitrarily powerful alignment maps, causal abstraction becomes vacuous: any neural network can be perfectly mapped to any algorithm, a result proven formally in Theorem 1 under five mild assump
- The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets
At sufficient scale, LLMs linearly represent the truth or falsehood of factual statements in their internal activations — a claim supported by PCA visualizations, cross-dataset probe transfer, and cau
- Endogenous Resistance to Activation Steering in Language Models
- Mechanistic Interpretability of EEG Foundation Models via Sparse Autoencoders
Applying TopK Sparse Autoencoders (SAEs) to three architecturally distinct EEG foundation models — SleepFM, REVE, and LaBraM — reveals that clinical concepts are not cleanly separable in these models'
- Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior
Manifold steering — intervening on model activations along paths constrained to lie on a learned activation manifold M_h rather than along Euclidean linear directions — produces behavioral trajectorie