concept
active
concept:sae-sparse-features-100k-features-64-active-per-tokenSAE sparse features (100K+ features, 64 active per token)
The specific SAE architecture trained: 100K+ possible features compressed to 64 active per token for layer-40 activations
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Fig 2 right.
- Interpretability tool underlying top-feature, details, steering tools.
- Basic SAE performance metrics.
- Table 1.
- Special tokens have very large magnitude, so their SAE features also have large magnitude and sit far from other pointshypothesis0.782First explanation for isolated features.
- Interpretability method criticized in this paper for shattering manifolds into atomic pieces, obscuring overarching semantic structure.
- Most features dead in largest SAE, indicating room for improvement.
- Standard baseline SAE operating on individual token activations.