question
active
question:this-raises-questions-about-the-scale-dependence-of-sae-evaluation-metrics-themselvesThis raises questions about the scale-dependence of SAE evaluation metrics themselves
Open issue from Pythia vs Gemma.
Source paper
extracted_from(2025) · Adam Karvonen · Can Rager · Johnny Lin · Curt Tigges +11
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Smaller models' weaker concept representations explain why supervised metrics differ on Pythia-160MgatesInterpretation of model-scale effects.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Design philosophy.
- Log-loss recovery ratio between SAE-inserted and ablated model.
- Sweeping number of features and training steps to find compute-optimal SAE configurations.
- Surprising finding that the two evaluation methods diverge in their relationship with persistence
- Backprop of logit-minus-mean-logit loss to surface causal intervention candidates, summed across token positions.
- Extension of mechanistic interpretability findings to the metacognitive domain
- Forward-looking claim about the potential of model introspection as an interpretability tool
- Left open.