concept
active
concept:weight-superpositionWeight superposition
Virtual weights between features forced into superposition by low-dimensional projections.
Neighborhood — ranked by edge-count
Papers (1)
paper
Concepts (2)
concept
- Superpositionextendsrelated_toPhenomenon where models represent more features than dimensions via almost-orthogonal directions.
- Interference Weightsassociated_withLogit weight contributions from a feature that arise due to superposition with other features, not from the feature's own causal role
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- More computations than neurons/dimensions.
- Attentional features spread across multiple heads.
- Any method enabling enumeration of features.
- Superposition configuration associated with a plateau and a dip in vulnerability.
- Core theoretical framework: neural networks represent more features than neurons by encoding features as directions in superposition
- Specific phrases or sequences memorized via binary features in superposition, enabling narrow pattern matching despite few neurons
- Baseline MTL approach minimizing sum of task losses with equal weights; suffers from task balancing
- The space of the model's parameter matrices, where VPD operations take place.