concept
active
concept:attention-streamsAttention streams
Term introduced by this paper for the horizontal axis of information flow in transformers across token-positions via attention heads
Neighborhood — ranked by edge-count
Papers (1)
paper
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Selective attention mechanism is a component of GWT-2 indicator; AST-1 models and enables control over attention
- Process using Q, K, V to compute a heat map over K and weighted sum of V.
- Study of what attentional features and head behaviors exist.
- An infinite sequence data type with semantics as functions from Nat to a.
- Proposed attention analog of sparse monosemantic features for decomposing attention layers.
- Blocking attention edges to trace information flow.
- Formal model of meditation phenomenology: focus → distraction → awareness of distraction → redirection, derived from active inference.
- Phenomenon where layers concentrate most attention weight on the first token, captured by ColSum Concentration metric