hypothesis
active
hypothesis:feature-splitting-determines-scaling-behavior-on-disentanglement-metricsFeature splitting determines scaling behavior on disentanglement metrics
Explains inverse scaling.
Source paper
extracted_from(2025) · Adam Karvonen · Can Rager · Johnny Lin · Curt Tigges +11
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (2)
finding
- Absorption scores worsen with dictionary size for all architectures except Matryoshka, which degrades only slightlyassociated_withInverse scaling.
- Rules out fraction-modified explanation.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Metrics worsen with dictionary size for most architectures.
- Matryoshka's hierarchical structure explains its positive scaling on disentanglement metricsclaim0.782Mechanistic interpretation.
- Independent concepts are encoded by separate latents.
- Authors argue the absence of a fixed feature count is a property of the superposition geometry, not a failure of the method
- Observed across SAE scales, e.g., 'San Francisco' split into 11 features.
- Conclusion.
- Main proposal.
- Discrete case.