framework
active
framework:isotropic-superposition-modelIsotropic Superposition Model
Prior model of superposition where features are discrete 1D objects repelling each other roughly evenly; paper argues this is incomplete
Neighborhood — ranked by edge-count
Claims (1)
claim
- Authors revise their own prior Toy Models framework based on evidence from feature splitting and geometry
Frameworks (1)
framework
- Feature ManifoldsextendsHypothesized extension of superposition where features may be higher-dimensional manifolds rather than 1D directions
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Open question.
- Core theoretical framework: neural networks represent more features than neurons by encoding features as directions in superposition
- Phenomenon where models represent more features than dimensions via almost-orthogonal directions.
- Synthetic benchmark with known ground-truth manifolds.
- Virtual weights between features forced into superposition by low-dimensional projections.
- Because superposition is a phase, there may exist a regime for every model with no superpositionclaim0.751Cause for hope.
- Cost of removing superposition.
- Representation of features spread across multiple layers, complicating dictionary learning.