concept
active
concept:superpositionSuperposition
Phenomenon where models represent more features than dimensions via almost-orthogonal directions.
Neighborhood — ranked by edge-count
Papers (9)
paper
- Toy Models of Superpositionintroducesmentions
- Zoom In: An Introduction to Circuitsintroduces
Thinkers (3)
thinker
- Gabriel GohstudiesReported multimodal neurons in vision models including region neurons and person detector neurons
- Arora et al.studiesEarliest superposition reference in word embeddings.
- Cheung et al.studiesModel-level superposition.
Frameworks (1)
framework
- Compressed Sensingassociated_withMathematical framework enabling recovery of high-dimensional sparse vectors from low-dimensional projections; theoretically underpins sparse autoencoder approach
Methods (2)
method
- Replacement model architecture with 30M features replacing MLP neurons.
- Jacobian matchingassociated_withPenalizing difference between transcoder and MLP Jacobians, computed cheaply via random vector products.
Concepts (10)
concept
- Weight superpositionextendsrelated_toVirtual weights between features forced into superposition by low-dimensional projections.
- Attention superpositionrelated_toAttentional features spread across multiple heads.
- Computation in superpositionrelated_toMore computations than neurons/dimensions.
- Polysemantic Neuronassociated_withA neuron that responds to multiple unrelated inputs, posing a major challenge for circuit-level interpretation
- Cross-layer superpositionextendsRepresentation of features spread across multiple layers, complicating dictionary learning.
- Circuit MotifextendsA recurring, abstract pattern found in circuits (e.g., equivariance, unioning over cases), inspired by circuit motifs in systems biology
- Feature Sparsityassociated_withProperty that features activate on only a small fraction of inputs; enables compressed sensing and is what allows superposition to work
- Interference between featuresassociated_withCost of non-orthogonal feature embedding.
- Privileged Basisassociated_withA property of activations where neural network features align with basis dimensions due to sparse activation functions; absent in the residual stream but present in tokens, attention patterns, and MLP activations
- Principal Component Analysisanalogous_toLinear-model solution that trades off with superposition.
Questions (1)
question
- Open question.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Core theoretical framework: neural networks represent more features than neurons by encoding features as directions in superposition
- Theoretical model of how neural networks encode more features than dimensions, informing linear representation work.
- Specific phrases or sequences memorized via binary features in superposition, enabling narrow pattern matching despite few neurons
- Any method enabling enumeration of features.
- The state in which a dialogue agent maintains multiple possible characters simultaneously, refined as the conversation proceeds
- Superposition configuration associated with a plateau and a dip in vulnerability.
- Graph with edges (Wi·Wj)^2 to visualize interference.
- superposition is deeply linked to the ability to identify and enumerate over all features in a modelquote0.803Safety rationale.