concept
active
concept:interpretability-in-the-wild-a-circuit-for-indirect-object-identification-in-gpt-2-small-wang-et-al-2023Interpretability in the Wild: A Circuit for Indirect Object Identification in GPT-2 Small (Wang et al., 2023)
Cited as causal intervention methodology precedent for this paper's ablation approach
Neighborhood — ranked by edge-count
Papers (1)
paper
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Importance of recursive generation.
- Central thesis of the post.
- GPT-2 implements at least one induction head using pointer arithmetic on positional embeddings rather than K-compositionhypothesis0.751Observation of an alternative induction head implementation algorithm in larger models with positional embeddings in the residual stream
- Simple DoM vectors coherently represent reward hacking and are generalizable and interpretableclaim0.749Central claim.
- Future work.
- Simple interpretability methods scale to 2.8T-parameter models and long-context agentic rolloutsclaim0.747Scaling claim.
- Motivation for VPD's parameter-focused approach.
- A better basis (e.g., factoring tokens into languages or parts of speech) may yield a sparse interpretable modelhypothesis0.745Same model can be dense in one basis and sparse in another.