finding
active
finding:adding-sae-error-term-reduces-ravel-disentangle-by-0-02-so-omittedAdding SAE error term reduces RAVEL Disentangle by ~0.02, so omitted
Design decision.
Source paper
extracted_from(2025) · Adam Karvonen · Can Rager · Johnny Lin · Curt Tigges +11
Neighborhood — ranked by edge-count
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Design property of the intervention method.
- The objective function combining L2 reconstruction error and L1 penalty scaled by decoder norm, used to train the SAE.
- Table 1.
- Interpretability method criticized in this paper for shattering manifolds into atomic pieces, obscuring overarching semantic structure.
- Lower SAE loss at fixed sparsity likely indicates less interference and less superposition.claim0.737Justifies SAE proxy.
- Claim that feature grounding enables interpretability metrics.
- Fig 1.
- Computing attribution as the dot product of the output logit gradient with the SAE decoder weight, multiplied by feature activation.