question
active
question:are-saes-in-the-pathological-scaling-regime-in-practiceare SAEs in the pathological scaling regime in practice?
Left open.
Source paper
extracted_from(2025) · Michaud, Eric J. · Gorton, Liv · McGrath, Tom
Neighborhood — ranked by edge-count
Papers (1)
paper
- Understanding sparse autoencoder scaling in the presence of feature manifoldsassociated_withmentions
Findings (1)
finding
- Strongest argument against pathology.
Claims (1)
claim
- Core concern.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Uncertain α and β.
- SAEs learn far fewer features than latents.
- Claim that feature grounding enables interpretability metrics.
- Loss follows a power law in the number of latents.
- Sweeping number of features and training steps to find compute-optimal SAE configurations.
- Link to prior work.
- Open issue from Pythia vs Gemma.
- Forward-looking claim about the potential of model introspection as an interpretability tool