concept
active
concept:solving-superpositionSolving superposition
Any method enabling enumeration of features.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Toy Models of Superpositionmentions
Thinkers (1)
thinker
- Sharkey et al.studiesFollowed up with sparse autoencoder extraction.
Claims (1)
claim
- Strategic picture.
Methods (1)
method
- Removes superposition at cost of loss.
Concepts (1)
concept
- Candidate approach to solving superposition.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- More computations than neurons/dimensions.
- Phenomenon where models represent more features than dimensions via almost-orthogonal directions.
- Open question.
- Attentional features spread across multiple heads.
- Core theoretical framework: neural networks represent more features than neurons by encoding features as directions in superposition
- Specific phrases or sequences memorized via binary features in superposition, enabling narrow pattern matching despite few neurons
- Virtual weights between features forced into superposition by low-dimensional projections.
- Theoretical model of how neural networks encode more features than dimensions, informing linear representation work.