hypothesis
active
hypothesis:superposition-and-polysemanticity-increase-as-feature-sparsity-increasesSuperposition and polysemanticity increase as feature sparsity increases
Major prediction checked against InceptionV1.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Toy Models of Superpositionmentions
Findings (1)
finding
- ReLU hidden layer model (n=10, m=5) shifts from monosemantic to polysemantic neurons as sparsity increasesassociated_withNeuron-level phase change.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Most pragmatically important question.
- Strategic Approach 1.
- Main proposal.
- Central interpretation.
- Lower SAE loss at fixed sparsity likely indicates less interference and less superposition.claim0.793Justifies SAE proxy.
- A head can implement multiple behaviors via high-rank output without superposition.
- Superposition begins with less important features in antipodal pairs, then transitions to other geometriesfinding0.788Toy model sparsity sweep.
- Core mechanism statement.