claim
active
claim:this-note-is-the-first-demonstration-of-an-interference-weight-inside-a-trained-transformer-by-measuring-effect-on-training-lossThis note is the first demonstration of an interference weight inside a trained transformer by measuring effect on training loss
Novelty claim.
Source paper
extracted_fromNicholas L. Turner · Jeffrey Wu · Joshua Batson
Neighborhood — ranked by edge-count
Papers (1)
paper
Findings (1)
finding
- Largest IN Tokens→Logits weight votes 'utions', which never follows IN in training; Fisher effectiveness ~3 orders below the top weights.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Concluding statement.
- Transformers almost surely maintain input-injectivity throughout training, not just at initialisationhypothesis0.786Conjecture supported by Nikolaou et al. 2025 for last-token hidden states
- Interpretation of motivating example.
- Monitoring use case.
- Logit weight contributions from a feature that arise due to superposition with other features, not from the feature's own causal role
- Conclusion.
- Identifying interference weights is necessary but not sufficient for reading global circuitsclaim0.748Discussion.
- Do we assume that the model knows something we don't, or are these interference weights contributing noise?question0.739Motivating question.