finding
active
finding:gemmascope-saes-significantly-outperform-mlp-neuron-pca-and-residual-stream-baselines-on-autointerpGemmaScope SAEs significantly outperform MLP neuron, PCA and residual stream baselines on autointerp
Figure 5.
Source paper
extracted_from(2025) · Adam Karvonen · Can Rager · Johnny Lin · Curt Tigges +11
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Design philosophy.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Scaling trends at larger scale.
- Quantitative comparison supporting SAE utility.
- SAEs trained on pretrained Gemma-2 models used for steering in Gemma family experiments
- SAE features are not simply mirroring individual neurons.
- Table 1.
- Probing limited.
- Gemma-2-27B attention layer Latent SOO MSE reduced from 11 to 7.67 ± 0.77 after SOO fine-tuningfinding0.759SOO fine-tuning reduced attention layer MSE in Gemma-2-27B though MLP layers showed no significant change
- Experiment 2 result showing large Gemma model supports high-dimensional truth cones