paper:doi-10-48550-arxiv-2503-09532SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
TL;DR
Gains on the sparsity-fidelity frontier do not predict how well a sparse autoencoder performs on downstream tasks, and SAEBench demonstrates this by scoring over 200 SAEs on eight metrics. Matryoshka BatchTopK SAEs rank below TopK and BatchTopK on loss recovered, yet they take the best score on Absorption, RAVEL, Sparse Probing, SCR and TPP, which is 5 of 8 metrics, in the typical L0 range of 40-200. Matryoshka is also the only architecture whose disentanglement scores improve when width grows from 4k to 16k to 65k latents on Gemma-2-2B (layer 12). The other architectures show inverse scaling on absorption and SCR, and the effect persists when the number of ablated latents is held fixed. SAEBench combines four capability groups (concept detection, interpretability, reconstruction, feature disentanglement). It adds two novel zero-ablation metrics, Spurious Correlation Removal and Targeted Probe Perturbation, and it extends the absorption metric to partial and shared absorption. The suite covers seven architectures (ReLU, TopK, BatchTopK, Gated, JumpReLU, P-annealing, Matryoshka), Gemma-2-2B and Pythia-160M, and Gemma-Scope SAEs up to 1M latents. Hierarchical architectures beat the rest by 30-40% on SCR and absorption. No single L0 wins everywhere, though L0 of 50-150 is a reasonable compromise. The implication is that SAE progress needs a multi-metric benchmark, and that optimizing reconstruction at fixed sparsity can hide failures in concept isolation, which grow with scale. The paper declines to combine the metrics into one score.
What to take away
- 1. Matryoshka BatchTopK SAEs rank worse than TopK and BatchTopK on the Gemma-2-2B 65k sparsity-fidelity frontier yet achieve the best scores on 5 of 8 SAEBench metrics (Absorption, RAVEL, Sparse Probing, SCR, TPP) in the L0 range 40-200.
- 2. Matryoshka is the only one of seven architectures whose feature disentanglement improves as dictionary width scales from 4k to 16k to 65k, while absorption and SCR scores worsen for the others.
- 3. The ReLU SAE is outperformed by the other architectures on 5 of 8 metrics, but 65k ReLU SAEs with L0 above 200 score best on 1-sparse probing.
- 4. Hierarchical architectures beat non-hierarchical ones on SCR and absorption by margins of 30-40%, which the authors present as clear differentiation.
- 5. SAEBench introduces Spurious Correlation Removal and Targeted Probe Perturbation, which use zero ablation of k latents (main results at k=20, with trends consistent for k from 5 to 50) to test whether a concept is complete in a small set of latents.
- 6. The inverse scaling on SCR is not an artifact of a fixed ablation count, because non-hierarchical architectures still degrade at 65k versus 16k when the intervention size k is varied over 5 to 500.
- 7. Over a sweep of L0 from 20 to 1000, no single sparsity is best, but moderate L0 of 50-150 balances the metrics, while TPP and RAVEL prefer L0 above 400.
- 8. The authors hypothesize that feature splitting drives inverse scaling on disentanglement metrics and that Matryoshka's hierarchical design avoids it, but they do not test this mechanism directly.
- 9. Methodologically, the absorption metric replaces the ablation-effect criterion of Chanin et al. with a probe-direction projection criterion (tau_fs=0.03), so that it works at later layers, and it also flags partial and multi-latent absorption.
- 10. Pythia-160M does not reproduce the Gemma-2-2B advantage of hierarchical SAEs on SCR, which raises the open question of whether supervised SAE metrics depend on model scale, and 94% residual probing versus 84% for a randomly initialized Pythia-1B shows how weak the supervised-concept signal is in poorly trained models.
Peer brief — for seminar discussion
SAEBench is a standardized suite of eight evaluations for sparse autoencoders, grouped into concept detection (k-sparse probing, feature absorption), interpretability (LLM-judged detection with gpt4o-mini), reconstruction (loss recovered), and feature disentanglement (unlearning on WMDP-bio, RAVEL, and two new metrics, Spurious Correlation Removal and Targeted Probe Perturbation). The team trained over 200 SAEs across seven architectures at widths of 4k, 16k and 65k and L0 from 20 to 1000, on Gemma-2-2B layer 12 and Pythia-160M layer 8, and also scored Gemma-Scope SAEs up to 1M latents on Gemma-2-2B and 9B. The load-bearing finding is that the sparsity-fidelity frontier is a poor predictor of downstream quality. Matryoshka BatchTopK sits mid-pack on loss recovered but wins 5 of 8 metrics at L0 of 40-200, and it is the only architecture that does not show inverse scaling on absorption and SCR, with hierarchical designs ahead by roughly 30-40%. The implication is that SAE architecture work tuned for reconstruction at fixed sparsity can hide worse concept isolation, especially at larger widths, and that practitioners should evaluate across several L0 values, with 50-150 a reasonable compromise. The paper's stated hypothesis is that feature splitting causes the inverse scaling and that Matryoshka's multi-level nesting prevents it. A critical reader would push back on several points. The supervised metrics cover only a handful of concepts (profession, gender, first letters, sentiment), so the headline ranking may reflect those concepts rather than SAE quality in general. The mechanism is argued from prior work rather than tested, for example with meta-SAEs or absorption counts per width. Scaling is done at fixed training tokens (500M), so larger SAEs are undertrained relative to compute, which could produce the inverse scaling. The Pythia-160M results do not replicate the Matryoshka SCR advantage, which weakens the generality claim. The benchmark could instead have used an alternative such as ground-truth supervised dictionaries as in Makelov et al. or SAGE, or board-game models with known features, which would avoid depending on proxy probes; the authors chose scalable probe-based metrics instead. The paper also declines to produce a single aggregate score, which keeps trade-offs visible but makes leaderboard-style comparison harder.
Methods (15)
- Automated Interpretability (LLM judge)Detection-based LLM judging of latent descriptions using gpt-4o-mini.
- Feature Absorption metric (extended)Adapted absorption metric detecting partial and shared absorption across layers.
- k-Sparse ProbingConcept detection metric using top-k latents and linear probe.
- Loss RecoveredReconstruction fidelity score based on cross-entropy.
- Maximum mean difference latent selectionFeature selection for sparse probing.
- Multitask Differentiable Binary Mask (MDBM)Used to select latents in RAVEL.
- Multitask Distributed Alignment Search (MDAS)Skyline for RAVEL achieving 0.87.
- PCA BaselinePCA components treated as SAE latents for comparison.
- RAVEL evaluationDisentanglement metric via latent interventions with Cause and Isolation metrics.
- SHIFTMethod from sparse feature circuits that SCR automates.
- Sparse Autoencoders (SAEs)Unsupervised baseline method for decomposing activations into sparse linear combinations; NLAs positioned as complementary approach.
- Spurious Correlation Removal (SCR)Novel metric extending SHIFT via zero ablation.
- Targeted Probe Perturbation (TPP)Novel multiclass generalization of SCR.
- Unlearning evaluationConditional negative clamping on WMDP-bio with MMLU retention.
- Zero ablation of latentsStronger test of concept isolation than steering.
Frameworks (8)
- BatchTopK SAEEvaluated architecture.
- Gated SAEEvaluated architecture.
- JumpReLU SAEEvaluated architecture.
- Matryoshka BatchTopK SAEHierarchical architecture that excels on disentanglement and scales positively.
- P-Annealing SAEEvaluated architecture.
- ReLU SAEBaseline SAE architecture.
- SAEBenchEvaluation suite of eight metrics across four capabilities.
- TopK SAEEvaluated architecture (Gao et al.).
Datasets (14)
- AG NewsTopic probing.
- Amazon ReviewsUsed for probing, SCR, TPP.
- Bias in BiosUsed for sparse probing, SCR, TPP.
- EuroparlLanguage ID probing.
- Gemma-2-2B layer 12 activationsMain model for SAE suite.
- Gemma-Scope SAEsOpen SAEs evaluated at widths 16k-1M.
- GitHub code datasetProgramming language probing.
- MMLUBenchmark used to evaluate performative reasoning; shows significantly more performative reasoning than GPQA-Diamond (easier task).
- OpenWebTextDataset used to measure language model performance (bits-per-byte) for cross-modal alignment experiments
- Pythia-160M layer 8 activationsSecond model for SAE suite.
- RAVEL datasetCities and Nobel laureates with attributes.
- The PileTraining corpus used for the 67M-parameter model tested with VPD.
- WikiTextUnlearning retain set.
- WMDP-bioUnlearning forget set/test.
Findings (26)
- SAEs on trained Pythia-1B significantly outperform SAEs on random model on sparse probing and SCR; random-model SAEs match or underperform residual probing
Supervised metrics discriminate.
- Matryoshka Batch TopK SAEs perform best on 5 of 8 metrics in the L0 40-200 range on 65k Gemma-2-2B despite worse sparsity-fidelity than TopK/BatchTopK
Central result.
- Gemma-Scope: loss recovered and autointerp improve with width; absorption, SCR, TPP degrade at larger widths; unlearning near zero at final layer
Scaling trends at larger scale.
- At L0<100 sparsity-fidelity ranking is BatchTopK > TopK > Jump > Gated > Matryoshka > p-anneal > ReLU; differences shrink at L0 100-500
Frontier ranking differs from other metrics.
- Moderate L0 of 50-150 balances metrics; lower L0 helps interpretability; higher L0 improves reconstruction, RAVEL, TPP and absorption
Task-dependent optimal sparsity.
- K-sparse probing gives limited differentiation; all SAEs outperform residual-channel probing baseline (0.65) on Gemma-2-2B layer 12
Probing limited.
- ReLU SAE is outperformed by all other architectures on absorption, SCR, RAVEL, TPP and loss recovered on Gemma-2-2B 65k
ReLU underperforms.
- SCR at fixed intervention budget decreases with width for most architectures while Matryoshka improves on Gemma-2-2B
Inverse scaling on SCR.
- Automated interpretability and Loss Recovered increase with dictionary size from 4k to 65k across all architectures
Positive scaling.
- Absorption scores worsen with dictionary size for all architectures except Matryoshka, which degrades only slightly
Inverse scaling.
Claims (12)
- Zero ablation is a stronger test of concept isolation than steering because only identifying all latents removes the concept
Justifies SCR/TPP.
- Downstream task performance and feature disentanglement are important considerations beyond the sparsity-fidelity trade-off
Conclusion.
- The best way to measure SAE quality is a range of diverse metrics on properties practitioners care about
Design philosophy.
- Smaller models' weaker concept representations explain why supervised metrics differ on Pythia-160M
Interpretation of model-scale effects.
- Supervised metrics are limited by ground-truth availability and cover only a fraction of latents
Limitation.
- Gains on unsupervised proxy metrics do not reliably translate to better practical performance
Core thesis.
- Matryoshka's hierarchical structure explains its positive scaling on disentanglement metrics
Mechanistic interpretation.
- Ablation set size K=20 is a practical size for manual analysis and trends hold for K in 5-50
Design choice.
- Metrics cannot be meaningfully combined into a single score without arbitrary weighting
Limitation.
- The sparsity-fidelity curve is necessary but not sufficient for evaluating SAEs
Motivates multi-metric evaluation.
Hypotheses (3)
- Inverse SCR scaling is simply due to a fixed number of ablated latents being a smaller fraction of larger SAEs
Alternative hypothesis tested and rejected.
- Higher sparsity forces composition of multiple concepts into fewer active latents, harming disentanglement
Explains TPP/RAVEL preferring high L0.
- Feature splitting determines scaling behavior on disentanglement metrics
Explains inverse scaling.
Questions (2)
- This raises questions about the scale-dependence of SAE evaluation metrics themselves
Open issue from Pythia vs Gemma.
- How can we evaluate dictionary learning for language model interpretability?
Opening question.
Original abstract (expand)
Sparse autoencoders (SAEs) are a popular technique for interpreting language model activations, and there is extensive recent work on improving SAE effectiveness. However, most prior work evaluates progress using unsupervised proxy metrics with unclear practical relevance. We introduce SAEBench, a comprehensive evaluation suite that measures SAE performance across eight diverse metrics, spanning interpretability, feature disentanglement and practical applications like unlearning. To enable systematic comparison, we open-source a suite of over 200 SAEs across eight recently proposed SAE architectures and training algorithms. Our evaluation reveals that gains on proxy metrics do not reliably translate to better practical performance. For instance, while Matryoshka SAEs slightly underperform on existing proxy metrics, they substantially outperform other architectures on feature disentanglement metrics; moreover, this advantage grows with SAE scale. By providing a standardized framework for measuring progress in SAE development, SAEBench enables researchers to study scaling trends and make nuanced comparisons between different SAE architectures and training methodologies. Our interactive interface enables researchers to flexibly visualize relationships between metrics across hundreds of open-source SAEs at: www.neuronpedia.org/sae-bench
Related work— refs + corpus + external arXiv
Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.
- Sparse Autoencoder Features for Classifications and TransferabilityShan Chen, Kuleen Sasse, Hugo Aerts, Thomas Hartvigsen, Danielle S. Bitterman Jack Gallifant2026≈ 84%
- Mechanistic Interpretability of Antibody Language Models Using SAEsOliver M. Turnbull, Anisha Parsan, Nithin Parsan, John J. Yang, Anna L. Beukenhorst, Charlotte M. Deane Rebonto Haque2026≈ 84%
- ≈ 83%
- Train One Sparse Autoencoder Across Multiple Sparsity Budgets to Preserve Interpretability and AccuracyYaroslav Aksenov, Daniil Laptev, Vadim Kurochkin, Gleb Gerasimov, Nikita Koryagin, Daniil Gavrilov Nikita Balagansky2025≈ 83%
- Rethinking Sparse Autoencoders: Select-and-Project for Fairness and Control from Encoder Features AloneCristian Daniel P\u{a}duraru, Teodor Poncu, Alexandru Tifrea, Elena Burceanu Antonio B\u{a}rb\u{a}lau2025≈ 83%
- Revisiting End-To-End Sparse Autoencoder Training: A Short Finetune Is All You NeedAdam Karvonen2025≈ 83%
- Resurrecting the Salmon: Rethinking Mechanistic Interpretability with Domain-Specific Sparse AutoencodersMudith Jayasekara, Max Kirkby Charles O'Neill2025≈ 82%
- SCALAR: Benchmarking SAE Interaction Sparsity in Toy LLMsAndrew Gordon, Peter Lai, Xavier Poncini, David Quarel, and Stefan Heimersheim Sean P. Fillingham2025≈ 82%
- Sparse Autoencoder Decomposition of Clinical Sequence Model Representations: Feature Complexity, Task Specialisation, and Mortality PredictionFeng Dong, Andreas Karwath Chris Sainsbury2026≈ 82%
- ≈ 82%
- MetaSAEs: Joint Training with a Decomposability Penalty Produces More Atomic Sparse Autoencoder LatentsMatthew Levinson2026≈ 82%
- SoftSAE: Dynamic Top-K Selection for Adaptive Sparse AutoencodersMarcin Mazur, Jacek Tabor, Przemys{\l}aw Spurek Jakub St\k{e}pie\'n2026≈ 82%
- A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language ModelsXuansheng Wu, Haiyan Zhao, Daking Rai, Ziyu Yao, Ninghao Liu, Mengnan Du Dong Shu2025≈ 82%
- ≈ 82%
- ≈ 82%
- Improving Sparse Autoencoder with Dynamic AttentionJinsen Zhang, Dawei Su, Hui Huang Dongsheng Wang2026≈ 82%
- How Pruning Reshapes Features: Sparse Autoencoder Analysis of Weight-Pruned Language ModelsElies Segu\'i-Mas, Guillermina Tormo-Carb\'o Hector Borobia2026≈ 82%
- Stable and Steerable Sparse Autoencoders with Weight RegularizationOliver M. Crook Piotr Jedryszek2026≈ 82%
- Do Sparse Autoencoders Capture Concept Manifolds?in corpus2026≈ 80%
- ≈ 80%
- Mechanistic Knobs in LLMs: Retrieving and Steering High-Order Semantic Features via Sparse Autoencodersin corpus2026≈ 80%
- ≈ 80%
- Open Problems in Mechanistic Interpretabilityin corpus2025≈ 80%
- Interpreting Language Model Parametersin corpus2026≈ 80%
- ≈ 79%
- ≈ 78%
- ≈ 78%
- ≈ 78%
- ≈ 74%
- ≈ 71%
+22 more