claim
active
claim:saes-may-have-the-right-inductive-biases-for-deeper-layers-but-not-middle-or-earlier-layersSAEs may have the right inductive biases for deeper layers but not middle or earlier layers.
Speculation about layer dependence.
Source paper
extracted_from(2025) · Lubana, Ekdeep Singh · Rager, Can · Hindupur, Sai Sumedh R. · Costa, Valerie +12
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Layer choice discussion.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Extension of mechanistic interpretability findings to the metacognitive domain
- Claim that feature grounding enables interpretability metrics.
- Automated interpretability and specificity ratings show SAE features are clearer than MLP neurons.
- Lower SAE loss at fixed sparsity likely indicates less interference and less superposition.claim0.778Justifies SAE proxy.
- Preference steering effective only in mid-layer range though correlations hold across layersfinding0.775Suggests mid-layer reading circuitry.
- Methodological critique of prior work that fixed a single layer for truth probing.
- Unsupervised recovery of known manifolds.
- Robustness influences superposition, predicting lower SAE loss in more robust models.hypothesis0.769Popperian credit for surprising prediction.