claim
active
claim:saes-assume-sparsity-number-of-concepts-is-time-invariant-drawn-i-i-d-from-a-fixed-distributionSAEs assume sparsity (number of concepts) is time-invariant, drawn i.i.d. from a fixed distribution.
Corollary 4.1.1.
Source paper
extracted_from(2025) · Lubana, Ekdeep Singh · Rager, Can · Hindupur, Sai Sumedh R. · Costa, Valerie +12
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- Fig. 2a,e.
Claims (2)
claim
- Proposition 4.2.
- Proposition 4.1 central claim.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Motivates multi-metric evaluation.
- Claim that feature grounding enables interpretability metrics.
- Lower SAE loss at fixed sparsity likely indicates less interference and less superposition.claim0.779Justifies SAE proxy.
- Broadly, turn-averaged SAEs make interpretability techniques practical at long context lengths.quote0.778Abstract thesis.
- Abstract.
- Better bases may give sparsity.
- Inception-v1 SAE shows meaningful fraction of latents with very high nearest-neighbor cosinefinding0.765Fig 8; may be duplication.
- Specific tuning curve structure.