finding
active
finding:more-robust-resnet18-models-consistently-achieve-lower-topk-sae-reconstruction-loss-at-a-given-sparsity-levelMore robust ResNet18 models consistently achieve lower TopK SAE reconstruction loss at a given sparsity level.
Evidence in real models.
Source paper
extracted_from(2025) · Gorton, Liv · Lewis, Owen
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (2)
claim
- Central thesis.
- Lower SAE loss at fixed sparsity likely indicates less interference and less superposition.associated_withJustifies SAE proxy.
Hypotheses (1)
hypothesis
- Robustness influences superposition, predicting lower SAE loss in more robust models.associated_withsupportsPopperian credit for surprising prediction.
Questions (1)
question
- Posed on SAE results.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Quantifies SAE reconstruction damage.
- Supervised metrics discriminate.
- Increasing TopK sparsity drives synthetic SAEs through shattering, capture, then dilution regimesfinding0.788Phase diagram on support size and receptive field spread.
- Baseline.
- Central result.
- SAE reconstruction error direction and norm are largely linearly predictable from the activation vectorfinding0.763Errors are structured not random.
- Sparse dictionary learning method used to extract interpretable features from EEG transformer embeddings.
- ReLU underperforms.