claim
active
claim:from-an-mdl-standpoint-block-sparsity-affords-a-more-compact-description-of-activations-than-directional-saesFrom an MDL standpoint, block sparsity affords a more compact description of activations than directional SAEs.
Support-term savings.
Source paper
extracted_from(2026) · Thomas Fel · Matthew Kowal · Mozes Jacobs · Dron Hazra +21
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (4)
finding
- MDL result at 20% distortion.
- Reconstruction cannot rank featurizers.
- On DINOv3, classification and segmentation tolerate R²≈0.8, while depth needs R²≈0.9 to retain 95% accuracy.associated_withTask-dependent distortion floors.
- Featurizer-specific optima.
Claims (1)
claim
- Source of MDL advantage.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Authors' experience-based claim.
- Main proposal.
- Expected limitation.
- Higher sparsity forces composition of multiple concepts into fewer active latents, harming disentanglementhypothesis0.753Explains TPP/RAVEL preferring high L0.
- Main theoretical claim, Prop. 1.
- Lower SAE loss at fixed sparsity likely indicates less interference and less superposition.claim0.751Justifies SAE proxy.
- Better bases may give sparsity.
- SAEs assume sparsity (number of concepts) is time-invariant, drawn i.i.d. from a fixed distribution.claim0.745Corollary 4.1.1.