concept
active
concept:causal-maskingcausal masking
Attention restricted to previous tokens only, as in decoder-only models; leads to AR(ω)-like behaviour and no ordered phase
Neighborhood — ranked by edge-count
Concepts (1)
concept
- context windowimplementsFinite number of previous tokens used by autoregressive models to predict the next token; defines interaction range
Findings (1)
finding
- Application to transformer language models
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Attention mechanism with causal mask limiting each token's view to previous tokens; used in decoder-only transformers
- Whether an internal direction causally controls a target behavior, verified by intervention success
- Modification to transformer restricting keys and values to previous time-steps only, mimicking how an agent accumulates experiences.
- Mechanistic interpretability technique for locating factual associations, mentioned as future work direction.
- Method by Chan et al. 2022 for rigorously testing interpretability hypotheses via interventions
- A framework the paper uses alongside feature geometry to deepen mechanistic understanding of LMs
- Confound where naming injected concepts reflects direct logit effects rather than metacognitive awareness, raised by Morris & Plunkett
- Function determining the value of a variable based on its causal parents in an acyclic causal model.