claim
active
claim:downstream-task-performance-and-feature-disentanglement-are-important-considerations-beyond-the-sparsity-fidelity-trade-offDownstream task performance and feature disentanglement are important considerations beyond the sparsity-fidelity trade-off
Conclusion.
Source paper
extracted_from(2025) · Adam Karvonen · Can Rager · Johnny Lin · Curt Tigges +11
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (2)
finding
- Positive scaling.
- Scaling trends at larger scale.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Traditional proxy evaluation criterion for SAEs.
- Main proposal.
- Higher sparsity forces composition of multiple concepts into fewer active latents, harming disentanglementhypothesis0.769Explains TPP/RAVEL preferring high L0.
- Explains inverse scaling.
- Better bases may give sparsity.
- Playful and poetic amplification support.
- Discovery story.
- Mechanistic argument.