concept
active
concept:cross-layer-superpositionCross-layer superposition
Representation of features spread across multiple layers, complicating dictionary learning.
Neighborhood — ranked by edge-count
Papers (2)
paper
Thinkers (1)
thinker
- Yun et al.studiesFirst to track unsupervised features across layers with a single dictionary.
Frameworks (1)
framework
- Superposition HypothesisextendsCore theoretical framework: neural networks represent more features than neurons by encoding features as directions in superposition
Methods (1)
method
- Sparse CrosscoderstudiesSparse dictionary learning variant that reads and writes to multiple layers or models, yielding shared features.
Concepts (1)
concept
- SuperpositionextendsPhenomenon where models represent more features than dimensions via almost-orthogonal directions.
Hypotheses (1)
hypothesis
- Consistent but alternative explanations like amplification exist.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Features smeared across layers cannot be fully disentangled by SAE on a single residual stream.
- Theoretical model of how neural networks encode more features than dimensions, informing linear representation work.
- More computations than neurons/dimensions.
- Interpretation of eval loss result.
- superposition is deeply linked to the ability to identify and enumerate over all features in a modelquote0.779Safety rationale.
- Central thesis.
- The state in which a dialogue agent maintains multiple possible characters simultaneously, refined as the conversation proceeds
- Virtual weights between features forced into superposition by low-dimensional projections.