concept
active
concept:mlp-neuron-superpositionMLP neuron superposition
MLP neuron polysemanticity requires superposition, unlike heads.
Neighborhood — ranked by edge-count
Papers (1)
paper
Concepts (1)
concept
- Head polysemanticity without superpositionanalogous_toA head can implement multiple behaviors via high-rank output without superposition.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- The sparse set of 28 neurons at layer 18 identified as responsible for Fourier feature computation across all cyclic tasks
- Feed-forward neural network with hidden layers, capable of representing non-linearly separable functions.
- Theoretical model of how neural networks encode more features than dimensions, informing linear representation work.
- Superposition hypothesis: neural networks represent more features than dimensions using almost-orthogonal directions.hypothesis0.777Explanation for why dictionary learning can recover many more features than dimensions.
- Interpretation of the cars-in-superposition circuit finding as an intentional representational strategy
- Superposition hypothesis: networks represent more features than neurons as almost-orthogonal directionshypothesis0.762Central hypothesis.
- Major open problem identified in the paper; MLP layers constitute 2/3 of transformer parameters
- Zeroing MLP layers to localize task neurons.