claim
active
claim:models-significantly-benefit-from-superposition-effectively-making-models-much-biggerModels significantly benefit from superposition, effectively making models much bigger
Cost of removing superposition.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Toy Models of Superpositionmentions
Claims (1)
claim
- MoE models should have less superposition because they eat the gap between neuron and feature sparsityassociated_withStrategic Approach 1.
Questions (1)
question
- Open question.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- superposition is deeply linked to the ability to identify and enumerate over all features in a modelquote0.823Safety rationale.
- Distinguishes from undertraining.
- Interpretation of the cars-in-superposition circuit finding as an intentional representational strategy
- Main contribution for interpretability.
- Authors' hedged interpretation.
- Mathematical derivation from Saxe-style loss.
- Bigger models are more likely to converge to a shared representation than smaller modelshypothesis0.784Selective pressure toward convergence via model capacity
- Unexpected result.