finding
active
finding:simulated-sae-with-100m-latents-discovers-only-3m-features-whenSimulated SAE with 100M latents discovers only ~3M features when β≪α
Fig 2 right.
Source paper
extracted_from(2025) · Michaud, Eric J. · Gorton, Liv · McGrath, Tom
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- When β<α, SAEs can optimally tile common feature manifolds at the expense of discovering rarer featuresassociated_withsupportsCore concern.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- The specific SAE architecture trained: 100K+ possible features compressed to 64 active per token for layer-40 activations
- Training artifact.
- Basic SAE performance metrics.
- Inception-v1 SAE shows meaningful fraction of latents with very high nearest-neighbor cosinefinding0.789Fig 8; may be duplication.
- SAE Feature #77278 fires 195,040 times in corpus, associated with satisfaction vs. emptiness dimensionfinding0.783High-frequency SAE feature reported as controlling fundamental positive vs. negative affect dimension
- Most features dead in largest SAE, indicating room for improvement.
- Model diffing identifies a small, interpretable set of latents responsible for emergent misalignment
- Discrete case.