finding
active
finding:in-one-262k-gemma-scope-sae-10-100-latents-with-cosine-0-97-had-dead-nearest-neighborsIn one 262k Gemma Scope SAE, 10-100 latents with cosine>0.97 had dead nearest neighbors
Training artifact.
Source paper
extracted_from(2025) · Michaud, Eric J. · Gorton, Liv · McGrath, Tom
Neighborhood — ranked by edge-count
Papers (1)
paper
Hypotheses (1)
hypothesis
- Latents near digit/alphabet manifolds may have tiled a manifold with one killed to reduce L0supportsSpeculative explanation.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Gemma Scope decoder nearest-neighbor cosine similarities skewed above random but few extremely highfinding0.837Fig 7.
- Inception-v1 SAE shows meaningful fraction of latents with very high nearest-neighbor cosinefinding0.818Fig 8; may be duplication.
- Fig 2 right.
- Implies α≈0.5-0.7.
- Shows persona space axes are inherited from pre-training, not solely created by post-training
- Most features dead in largest SAE, indicating room for improvement.
- Model diffing identifies a small, interpretable set of latents responsible for emergent misalignment
- Experiment 4 result showing DIM captures only one facet of the multi-dimensional truth subspace