finding
active
finding:tfa-reaches-nmse-comparable-to-saes-on-gemma-2-2b-e-g-story-0-139-vs-batchtopk-0-152-web-0-139-vs-0-139-while-pred-only-is-worse-0-34-0-38TFA reaches NMSE comparable to SAEs on Gemma-2-2B (e.g., Story 0.139 vs BatchTopK 0.152; Web 0.139 vs 0.139), while Pred-only is worse (0.34–0.38).
Table 1.
Source paper
extracted_from(2025) · Lubana, Ekdeep Singh · Rager, Can · Hindupur, Sai Sumedh R. · Costa, Valerie +12
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- §7.2 assertion.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- TFA explains similar variance as SAEs (Story 0.73, Web 0.79, Code 0.75) versus Pred-only 0.29–0.40.finding0.848Table 2.
- Central result.
- Fig. 58.
- Table 1.
- Table 1.
- GemmaScope SAEs significantly outperform MLP neuron, PCA and residual stream baselines on autointerpfinding0.761Figure 5.
- Gemma-2-27B attention layer Latent SOO MSE reduced from 11 to 7.67 ± 0.77 after SOO fine-tuningfinding0.760SOO fine-tuning reduced attention layer MSE in Gemma-2-27B though MLP layers showed no significant change
- Probing limited.