finding
active
finding:34m-sae-had-roughly-65-dead-features34M SAE had roughly 65% dead features.
Most features dead in largest SAE, indicating room for improvement.
Source paper
extracted_from(2024) ·
Neighborhood — ranked by edge-count
Papers (1)
paper
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Basic SAE performance metrics.
- Moderation statistics.
- Fig 2 right.
- Training artifact.
- Empirical observation of feature splitting.
- The specific SAE architecture trained: 100K+ possible features compressed to 64 active per token for layer-40 activations
- SAE features are not simply mirroring individual neurons.
- Explains nested ≈ half-capacity turn-averaged SAE.