claim
active
claim:the-isotropic-superposition-model-is-incomplete-because-features-cluster-into-higher-density-groups-due-to-correlated-activations-and-similar-downstream-actionsThe isotropic superposition model is incomplete because features cluster into higher-density groups due to correlated activations and similar downstream actions
Authors revise their own prior Toy Models framework based on evidence from feature splitting and geometry
Source paper
extracted_from(2024) · Marc Carauleanu · Michael Vaiana · Judd Rosenblatt · Cameron Berg +1
Neighborhood — ranked by edge-count
Papers (1)
paper
- Toy Models of Superpositionextends
Frameworks (1)
framework
- Isotropic Superposition ModelcontradictsPrior model of superposition where features are discrete 1D objects repelling each other roughly evenly; paper argues this is incomplete
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- superposition is deeply linked to the ability to identify and enumerate over all features in a modelquote0.773Safety rationale.
- Confound addressed.
- Superposition hypothesis: neural networks represent more features than dimensions using almost-orthogonal directions.hypothesis0.764Explanation for why dictionary learning can recover many more features than dimensions.
- Superposition hypothesis: networks represent more features than neurons as almost-orthogonal directionshypothesis0.764Central hypothesis.
- Because superposition is a phase, there may exist a regime for every model with no superpositionclaim0.764Cause for hope.
- Core demonstration.
- Cost of removing superposition.
- Interpretation of the cars-in-superposition circuit finding as an intentional representational strategy