finding
active
finding:relu-sae-is-outperformed-by-all-other-architectures-on-absorption-scr-ravel-tpp-and-loss-recovered-on-gemma-2-2b-65k

ReLU SAE is outperformed by all other architectures on absorption, SCR, RAVEL, TPP and loss recovered on Gemma-2-2B 65k

ReLU underperforms.

Source paper

extracted_from
SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
(2025) · Adam Karvonen · Can Rager · Johnny Lin · Curt Tigges +11

Neighborhood — ranked by edge-count

Frameworks (1)

framework

Findings (1)

finding