paper
active
2025
paper:doi-10-48550-arxiv-2503-09532

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability

TL;DR

Gains on the sparsity-fidelity frontier do not predict how well a sparse autoencoder performs on downstream tasks, and SAEBench demonstrates this by scoring over 200 SAEs on eight metrics. Matryoshka BatchTopK SAEs rank below TopK and BatchTopK on loss recovered, yet they take the best score on Absorption, RAVEL, Sparse Probing, SCR and TPP, which is 5 of 8 metrics, in the typical L0 range of 40-200. Matryoshka is also the only architecture whose disentanglement scores improve when width grows from 4k to 16k to 65k latents on Gemma-2-2B (layer 12). The other architectures show inverse scaling on absorption and SCR, and the effect persists when the number of ablated latents is held fixed. SAEBench combines four capability groups (concept detection, interpretability, reconstruction, feature disentanglement). It adds two novel zero-ablation metrics, Spurious Correlation Removal and Targeted Probe Perturbation, and it extends the absorption metric to partial and shared absorption. The suite covers seven architectures (ReLU, TopK, BatchTopK, Gated, JumpReLU, P-annealing, Matryoshka), Gemma-2-2B and Pythia-160M, and Gemma-Scope SAEs up to 1M latents. Hierarchical architectures beat the rest by 30-40% on SCR and absorption. No single L0 wins everywhere, though L0 of 50-150 is a reasonable compromise. The implication is that SAE progress needs a multi-metric benchmark, and that optimizing reconstruction at fixed sparsity can hide failures in concept isolation, which grow with scale. The paper declines to combine the metrics into one score.

What to take away

  1. 1. Matryoshka BatchTopK SAEs rank worse than TopK and BatchTopK on the Gemma-2-2B 65k sparsity-fidelity frontier yet achieve the best scores on 5 of 8 SAEBench metrics (Absorption, RAVEL, Sparse Probing, SCR, TPP) in the L0 range 40-200.
  2. 2. Matryoshka is the only one of seven architectures whose feature disentanglement improves as dictionary width scales from 4k to 16k to 65k, while absorption and SCR scores worsen for the others.
  3. 3. The ReLU SAE is outperformed by the other architectures on 5 of 8 metrics, but 65k ReLU SAEs with L0 above 200 score best on 1-sparse probing.
  4. 4. Hierarchical architectures beat non-hierarchical ones on SCR and absorption by margins of 30-40%, which the authors present as clear differentiation.
  5. 5. SAEBench introduces Spurious Correlation Removal and Targeted Probe Perturbation, which use zero ablation of k latents (main results at k=20, with trends consistent for k from 5 to 50) to test whether a concept is complete in a small set of latents.
  6. 6. The inverse scaling on SCR is not an artifact of a fixed ablation count, because non-hierarchical architectures still degrade at 65k versus 16k when the intervention size k is varied over 5 to 500.
  7. 7. Over a sweep of L0 from 20 to 1000, no single sparsity is best, but moderate L0 of 50-150 balances the metrics, while TPP and RAVEL prefer L0 above 400.
  8. 8. The authors hypothesize that feature splitting drives inverse scaling on disentanglement metrics and that Matryoshka's hierarchical design avoids it, but they do not test this mechanism directly.
  9. 9. Methodologically, the absorption metric replaces the ablation-effect criterion of Chanin et al. with a probe-direction projection criterion (tau_fs=0.03), so that it works at later layers, and it also flags partial and multi-latent absorption.
  10. 10. Pythia-160M does not reproduce the Gemma-2-2B advantage of hierarchical SAEs on SCR, which raises the open question of whether supervised SAE metrics depend on model scale, and 94% residual probing versus 84% for a randomly initialized Pythia-1B shows how weak the supervised-concept signal is in poorly trained models.

Peer brief — for seminar discussion

SAEBench is a standardized suite of eight evaluations for sparse autoencoders, grouped into concept detection (k-sparse probing, feature absorption), interpretability (LLM-judged detection with gpt4o-mini), reconstruction (loss recovered), and feature disentanglement (unlearning on WMDP-bio, RAVEL, and two new metrics, Spurious Correlation Removal and Targeted Probe Perturbation). The team trained over 200 SAEs across seven architectures at widths of 4k, 16k and 65k and L0 from 20 to 1000, on Gemma-2-2B layer 12 and Pythia-160M layer 8, and also scored Gemma-Scope SAEs up to 1M latents on Gemma-2-2B and 9B. The load-bearing finding is that the sparsity-fidelity frontier is a poor predictor of downstream quality. Matryoshka BatchTopK sits mid-pack on loss recovered but wins 5 of 8 metrics at L0 of 40-200, and it is the only architecture that does not show inverse scaling on absorption and SCR, with hierarchical designs ahead by roughly 30-40%. The implication is that SAE architecture work tuned for reconstruction at fixed sparsity can hide worse concept isolation, especially at larger widths, and that practitioners should evaluate across several L0 values, with 50-150 a reasonable compromise. The paper's stated hypothesis is that feature splitting causes the inverse scaling and that Matryoshka's multi-level nesting prevents it. A critical reader would push back on several points. The supervised metrics cover only a handful of concepts (profession, gender, first letters, sentiment), so the headline ranking may reflect those concepts rather than SAE quality in general. The mechanism is argued from prior work rather than tested, for example with meta-SAEs or absorption counts per width. Scaling is done at fixed training tokens (500M), so larger SAEs are undertrained relative to compute, which could produce the inverse scaling. The Pythia-160M results do not replicate the Matryoshka SCR advantage, which weakens the generality claim. The benchmark could instead have used an alternative such as ground-truth supervised dictionaries as in Makelov et al. or SAGE, or board-game models with known features, which would avoid depending on proxy probes; the authors chose scalable probe-based metrics instead. The paper also declines to produce a single aggregate score, which keeps trade-offs visible but makes leaderboard-style comparison harder.

Methods (15)

Frameworks (8)

Datasets (14)

Findings (26)

Original abstract (expand)

Sparse autoencoders (SAEs) are a popular technique for interpreting language model activations, and there is extensive recent work on improving SAE effectiveness. However, most prior work evaluates progress using unsupervised proxy metrics with unclear practical relevance. We introduce SAEBench, a comprehensive evaluation suite that measures SAE performance across eight diverse metrics, spanning interpretability, feature disentanglement and practical applications like unlearning. To enable systematic comparison, we open-source a suite of over 200 SAEs across eight recently proposed SAE architectures and training algorithms. Our evaluation reveals that gains on proxy metrics do not reliably translate to better practical performance. For instance, while Matryoshka SAEs slightly underperform on existing proxy metrics, they substantially outperform other architectures on feature disentanglement metrics; moreover, this advantage grows with SAE scale. By providing a standardized framework for measuring progress in SAE development, SAEBench enables researchers to study scaling trends and make nuanced comparisons between different SAE architectures and training methodologies. Our interactive interface enables researchers to flexibly visualize relationships between metrics across hundreds of open-source SAEs at: www.neuronpedia.org/sae-bench

Related work— refs + corpus + external arXiv

Cited / in-corpus / arXiv badges show which signals surfaced each row. Multi-source rows weighted higher.

+22 more

Similar preprints — Semantic Scholar