finding
active
finding:logit-minus-mean-logit-loss-was-most-effective-for-attribution-vs-log-loss-which-was-confusing-to-interpretLogit-minus-mean-logit loss was most effective for attribution, vs log-loss which was confusing to interpret
Attribution loss selection.
Source paper
extracted_fromNeighborhood — ranked by edge-count
Papers (1)
paper
Methods (1)
method
- Backprop of logit-minus-mean-logit loss to surface causal intervention candidates, summed across token positions.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Compared to IMTL-L: parameter-free, no extra computational cost, achieves same theoretical goal
- Speculative explanation.
- Dot product of summed decoder vector with unembedding vector, following Hanna et al.
- Automated logit weight prediction achieves 74% mean accuracy for features vs 58% for neurons vs 50% chancefinding0.766Automated interpretability of logit weights confirms feature downstream effects are more interpretable than neuron effects
- Unclear reason stated by authors.
- Key quantitative evidence that detection signal is identical to global logit shift confound
- Logit attribution heatmaps and max-activating examples disagree for Features 425104 and 349410finding0.757Limitation.
- Quantifies the information gain from using logit-based expected value over greedy or sampled decoding