finding
active
finding:saes-on-trained-pythia-1b-significantly-outperform-saes-on-random-model-on-sparse-probing-and-scr-random-model-saes-match-or-underperform-residual-probingSAEs on trained Pythia-1B significantly outperform SAEs on random model on sparse probing and SCR; random-model SAEs match or underperform residual probing
Supervised metrics discriminate.
Source paper
extracted_from(2025) · Adam Karvonen · Can Rager · Johnny Lin · Curt Tigges +11
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- SAEs on random Pythia-1B achieve higher TPP than trained-model SAEs, attributed to lower probe baselineassociated_withCaveated result.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Random baseline context.
- Probing limited.
- Evidence in real models.
- Surprising negative result for LDA despite being a supervised method
- Case Study II result showing DAS identifies fewer causally relevant positions than a probe
- App. B.
- Attributed to model anisotropy from saturation making hidden states harder to access
- Authors' experience-based claim.