concept
active
concept:superposition-in-neural-networksSuperposition in Neural Networks
Theoretical model of how neural networks encode more features than dimensions, informing linear representation work.
Neighborhood — ranked by edge-count
Papers (1)
paper
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Open question.
- Phenomenon where models represent more features than dimensions via almost-orthogonal directions.
- Supported by abs model.
- Specific phrases or sequences memorized via binary features in superposition, enabling narrow pattern matching despite few neurons
- More computations than neurons/dimensions.
- MLP neuron polysemanticity requires superposition, unlike heads.
- Superposition hypothesis: neural networks represent more features than dimensions using almost-orthogonal directions.hypothesis0.805Explanation for why dictionary learning can recover many more features than dimensions.