question
active
question:are-there-any-models-of-superposition-which-have-a-closed-form-solutionAre there any models of superposition which have a closed-form solution?
Open question.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Toy Models of Superpositionmentions
Thinkers (1)
thinker
- Tom McGrathcites
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Any method enabling enumeration of features.
- Prior model of superposition where features are discrete 1D objects repelling each other roughly evenly; paper argues this is incomplete
- Phenomenon where models represent more features than dimensions via almost-orthogonal directions.
- Because superposition is a phase, there may exist a regime for every model with no superpositionclaim0.766Cause for hope.
- More computations than neurons/dimensions.
- If we don't give a model enough neurons to implement this solution, how does it do computation in superposition?question0.758Left to future work.
- Synthetic benchmark with known ground-truth manifolds.
- Cost of removing superposition.