finding
active
finding:dimensions-per-feature-d-is-sticky-at-1-and-1-2-in-n-400-m-30-uniform-superpositionDimensions per feature D* is sticky at 1 and 1/2 in n=400, m=30 uniform superposition
Resembles fractional quantum Hall effect.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Toy Models of Superpositionmentions
Claims (1)
claim
- Analogy to many ice phases.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Measures features learned.
- Perturbing a single feature.
- On DINOv3, classification and segmentation tolerate R²≈0.8, while depth needs R²≈0.9 to retain 95% accuracy.finding0.764Task-dependent distortion floors.
- Turn-averaged features have highest embedding similarity 0.663 at k=3; per-token max lowest at 0.617finding0.758Consistent across k=1,2,3,5.
- Measure of superposition in toy models.
- Core demonstration.
- Role of extra dimensions.
- Superposition hypothesis: neural networks represent more features than dimensions using almost-orthogonal directions.hypothesis0.746Explanation for why dictionary learning can recover many more features than dimensions.