method
active
method:gradient-based-attribution-to-sae-featuresGradient-based attribution to SAE features
Backprop of logit-minus-mean-logit loss to surface causal intervention candidates, summed across token positions.
Neighborhood — ranked by edge-count
Papers (1)
paper
- Understanding and Steering Llama 3 with Sparse Autoencoders - Goodfireintroducesmentions
Findings (2)
finding
- Attribution loss selection.
- Design choice validated empirically.
Methods (3)
method
- Gradient-based data attributionrelated_toBaseline method against which probe-based ranking is compared; more computationally expensive.
- Computing attribution as the dot product of the output logit gradient with the SAE decoder weight, multiplied by feature activation.
- Modifying SAE hidden activations h to h-tilde while keeping error term fixed, then continuing inference.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Used by APD; poor approximations of causal importance.
- Third checkpointing strategy; adopted here.
- Some gradient-based attribution methods give attributions independent of model and data-generating processfinding0.816Adebayo et al. sanity checks.
- A promising property for interpretability analysis off-distribution.
- Out-of-distribution generalization of SAE features.
- Interpretability tool underlying top-feature, details, steering tools.
- Motivation.
- Internal concepts a model uses, extracted as human-interpretable components.