finding
active
finding:matryoshka-batch-topk-saes-perform-best-on-5-of-8-metrics-in-the-l0-40-200-range-on-65k-gemma-2-2b-despite-worse-sparsity-fidelity-than-topk-batchtopkMatryoshka Batch TopK SAEs perform best on 5 of 8 metrics in the L0 40-200 range on 65k Gemma-2-2B despite worse sparsity-fidelity than TopK/BatchTopK
Central result.
Source paper
extracted_from(2025) · Adam Karvonen · Can Rager · Johnny Lin · Curt Tigges +11
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (2)
claim
- Gains on unsupervised proxy metrics do not reliably translate to better practical performancesupportsCore thesis.
- Motivates multi-metric evaluation.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Training dynamics.
- Frontier ranking differs from other metrics.
- Table 1.
- Model-scale dependence.
- Hierarchical architecture that excels on disentanglement and scales positively.
- Exception to ReLU underperformance.
- Evidence in real models.
- Increasing TopK sparsity drives synthetic SAEs through shattering, capture, then dilution regimesfinding0.750Phase diagram on support size and receptive field spread.