claim
active
claim:feature-splitting-means-dictionaries-with-fewer-features-provide-coarser-summaries-of-model-features-while-larger-dictionaries-reveal-finer-grained-distinctions-with-no-uniquely-correct-number-of-featuresFeature splitting means dictionaries with fewer features provide coarser summaries of model features while larger dictionaries reveal finer-grained distinctions, with no uniquely 'correct' number of features
Authors argue the absence of a fixed feature count is a property of the superposition geometry, not a failure of the method
Source paper
extracted_from(2024) · Marc Carauleanu · Michael Vaiana · Judd Rosenblatt · Cameron Berg +1
Neighborhood — ranked by edge-count
Findings (1)
finding
- Concrete example of feature splitting revealing unexpected model structure
Questions (1)
question
- what is the 'correct number of features' for dictionary learning, and is this question well-posed?gatesOpen question about whether there is a true discrete feature count or a continuous splitting process
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Observed across SAE scales, e.g., 'San Francisco' split into 11 features.
- Phenomenon where a feature in a small SAE splits into multiple finer features in a larger SAE.
- Problems suggesting sparsity breaks down as interpretability proxy.
- Explains inverse scaling.
- Primary unexpected empirical observation.
- Claimed by Braun et al.; demonstrated empirically here.
- Width buys features in order of utility, so only larger models learn infrequent or complex tasksclaim0.765Interpretation of Theorem 3.
- Motivation for using sparsity-based dictionary learning on language models