hypothesis
active
hypothesis:higher-sparsity-forces-composition-of-multiple-concepts-into-fewer-active-latents-harming-disentanglementHigher sparsity forces composition of multiple concepts into fewer active latents, harming disentanglement
Explains TPP/RAVEL preferring high L0.
Source paper
extracted_from(2025) · Adam Karvonen · Can Rager · Johnny Lin · Curt Tigges +11
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- Unexpected result.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Main proposal.
- Better bases may give sparsity.
- Major prediction checked against InceptionV1.
- Explains how top-down causation leverages lower-level problem-solving for large-scale goals.
- Conclusion.
- Support-term savings.
- SAEs assume sparsity (number of concepts) is time-invariant, drawn i.i.d. from a fixed distribution.claim0.751Corollary 4.1.1.
- Structured sparsity question.