finding
active
finding:first-order-phase-change-between-not-learned-superposition-and-dedicated-dimension-in-2-feature-1-dimension-and-3-feature-2-dimension-modelsFirst-order phase change between not-learned, superposition, and dedicated dimension in 2-feature 1-dimension and 3-feature 2-dimension models
Both empirical and theoretical phase diagrams.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Toy Models of Superpositionmentions
Claims (2)
claim
- Because superposition is a phase, there may exist a regime for every model with no superpositionsupportsCause for hope.
- The phase diagram explains why neurons are sometimes monosemantic and sometimes polysemanticsupportsTheory contribution.
Methods (1)
method
- Phase diagram experimentsupportsSweeps importance and sparsity to locate phase changes.
Artifacts (1)
artifact
- Closed-form losses for 2-feature and 3-feature configurations.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Load-bearing interpretive claim about the layer-specificity of Burger et al.'s finding.
- Perturbing a single feature.
- Analogy to many ice phases.
- Reinterpretation of Burger et al.'s finding as layer-specific rather than universal.
- Baseline.
- Core demonstration.
- Learning dynamics.
- Superposition begins with less important features in antipodal pairs, then transitions to other geometriesfinding0.774Toy model sparsity sweep.