method
active
method:residual-stream-saes-with-adjacent-layer-gradient-attributionsResidual stream SAEs with adjacent-layer gradient attributions
Third checkpointing strategy; adopted here.
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- Reason for adopting the third strategy.
Methods (2)
method
- Source of the features used in attributions.
- Strategy forcing edges to be mediated only by head paths of length 1 to avoid exponential path growth.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Backprop of logit-minus-mean-logit loss to surface causal intervention candidates, summed across token positions.
- Computing attribution as the dot product of the output logit gradient with the SAE decoder weight, multiplied by feature activation.
- Proposed pathway flowing through layers at each position; calculates K/V values that feed horizontal information flow.
- Architectural observation enabling the entire mathematical framework; the residual stream is purely a sum of linear projections
- Baseline method against which probe-based ranking is compared; more computationally expensive.
- The phenomenon where the residual stream communicates many more features than its dimensionality by encoding information across overlapping subspaces
- Used by APD; poor approximations of causal importance.
- The middle layer residual stream features are causally implicated in multi-step reasoning.claim0.762Features for Kobe Bryant, California, Lakers participate in computing the capital answer.