finding
active
finding:superposition-occurs-in-relu-output-toy-models-more-features-than-dimensions-represented-when-features-are-sparseSuperposition occurs in ReLU output toy models: more features than dimensions represented when features are sparse
Core demonstration.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Toy Models of Superpositionmentions
Claims (1)
claim
- Main contribution for interpretability.
Hypotheses (1)
hypothesis
- Superposition hypothesis: networks represent more features than neurons as almost-orthogonal directionsassociated_withsupportsCentral hypothesis.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Superposition controls robustness.
- Symmetry is not explained by the base toy model.
- superposition is deeply linked to the ability to identify and enumerate over all features in a modelquote0.795Safety rationale.
- Superposition hypothesis: neural networks represent more features than dimensions using almost-orthogonal directions.hypothesis0.793Explanation for why dictionary learning can recover many more features than dimensions.
- Interpretation of the cars-in-superposition circuit finding as an intentional representational strategy
- Figure 4.
- Confound addressed.
- Unexpected result.