question
active
question:if-we-don-t-give-a-model-enough-neurons-to-implement-this-solution-how-does-it-do-computation-in-superpositionIf we don't give a model enough neurons to implement this solution, how does it do computation in superposition?
Left to future work.
Neighborhood — ranked by edge-count
Papers (1)
paper
- A Toy Model of Mechanistic (Un)Faithfulnessassociated_withmentions
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Interpretation of the cars-in-superposition circuit finding as an intentional representational strategy
- Supported by abs model.
- Open question.
- More computations than neurons/dimensions.
- Because superposition is a phase, there may exist a regime for every model with no superpositionclaim0.783Cause for hope.
- Superposition hypothesis: neural networks represent more features than dimensions using almost-orthogonal directions.hypothesis0.782Explanation for why dictionary learning can recover many more features than dimensions.
- Open question.
- Data-induced competition over resources.