finding
active
finding:pipeline-retained-814-sae-feature-clusters-and-512-data-clusters-on-259-842-examplesPipeline retained 814 SAE feature clusters and 512 data clusters on 259,842 examples
App. B.
Source paper
extracted_from(2026) · Leon Bergen · Usha Bhalla · Sidharth Baskaran · Max Loeffler +13
Neighborhood — ranked by edge-count
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Fig 2 right.
- Unsupervised recovery of known manifolds.
- SAE Feature #77278 fires 195,040 times in corpus, associated with satisfaction vs. emptiness dimensionfinding0.737High-frequency SAE feature reported as controlling fundamental positive vs. negative affect dimension
- Feature cluster examples from the UMAP.
- Core critique of sparse autoencoders: they break the geometric structure of representations, making it harder to see the big picture.
- Most features dead in largest SAE, indicating room for improvement.
- The specific SAE architecture trained: 100K+ possible features compressed to 64 active per token for layer-40 activations
- Claim that feature grounding enables interpretability metrics.