claim
active
claim:lower-sae-loss-at-fixed-sparsity-likely-indicates-less-interference-and-less-superpositionLower SAE loss at fixed sparsity likely indicates less interference and less superposition.
Justifies SAE proxy.
Source paper
extracted_from(2025) · Gorton, Liv · Lewis, Owen
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- More robust ResNet18 models consistently achieve lower TopK SAE reconstruction loss at a given sparsity level.associated_withEvidence in real models.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Robustness influences superposition, predicting lower SAE loss in more robust models.hypothesis0.849Popperian credit for surprising prediction.
- Motivates multi-metric evaluation.
- Major prediction checked against InceptionV1.
- Hypothesized explanation.
- Proposition 4.2.
- Main proposal.
- SAEs assume sparsity (number of concepts) is time-invariant, drawn i.i.d. from a fixed distribution.claim0.779Corollary 4.1.1.
- Better bases may give sparsity.