framework
active
framework:superposition-hypothesisSuperposition Hypothesis
Core theoretical framework: neural networks represent more features than neurons by encoding features as directions in superposition
Neighborhood — ranked by edge-count
Papers (5)
paper
- Open Problems in Mechanistic Interpretabilityintroducesmentions
- Toy Models of Superpositionimplementsintroduces
Thinkers (2)
thinker
- Owen Lewisstudies
- Liv GortonstudiesAuthor at Goodfire; wrote group crosscoders paper.
Methods (3)
method
- Sparse CrosscoderstudiesSparse dictionary learning variant that reads and writes to multiple layers or models, yielding shared features.
- Sparse Dictionary LearningimplementsGeneral method for finding overcomplete sparse decompositions; the paper uses sparse autoencoders as an approximation
- Measure of superposition in toy models.
Concepts (9)
concept
- Polysemanticityassociated_withNeurons that respond to multiple unrelated concepts, limiting interpretability.
- Linear representationsupportsThe idea that features are encoded as directions in activation space.
- Cross-layer superpositionextendsRepresentation of features spread across multiple layers, complicating dictionary learning.
- When non-orthogonal features cause logistic regression to identify a suboptimal probe direction
- Feature SparsitysupportsProperty that features activate on only a small fraction of inputs; enables compressed sensing and is what allows superposition to work
- Sparse subframeextendsMathematical object defining J-space as union of cones.
- Memorization in Superpositionassociated_withSpecific phrases or sequences memorized via binary features in superposition, enabling narrow pattern matching despite few neurons
- Noisy Simulation of Sparse Networksassociated_withMechanism by which superposition works: small neural networks exploit sparsity to approximately simulate much larger sparse networks
- Overcomplete Basisassociated_withA set of feature directions that is larger than the dimensionality of the space, enabling superposition
Claims (2)
claim
- Motivates the introduction of mass-mean probing as an alternative to LR
- Authors' overall conclusion from number of interpretable features, activation-level correspondence to intensity, sensible logit weights, and interference weights
Frameworks (3)
framework
- Primary method introduced: trains a one-hidden-layer MLP with L1 sparsity penalty to decompose model activations into overcomplete feature dictionaries
- Compressed Sensingassociated_withMathematical framework enabling recovery of high-dimensional sparse vectors from low-dimensional projections; theoretically underpins sparse autoencoder approach
- DisentanglementcontradictsRelated research agenda seeking representations that separate conceptually distinct factors; contrasted with superposition approach
Findings (1)
finding
- Shows superposition enables many more features than neurons
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Phenomenon where models represent more features than dimensions via almost-orthogonal directions.
- Central thesis.
- More computations than neurons/dimensions.
- Supported by Table 1.
- Theoretical model of how neural networks encode more features than dimensions, informing linear representation work.
- Main contribution for interpretability.
- Any method enabling enumeration of features.
- Superposition hypothesis: networks represent more features than neurons as almost-orthogonal directionshypothesis0.791Central hypothesis.