concept
active
concept:key-query-attentionKey-query attention
Attention mechanism in AI used in Transformers; also proposed for GWT implementations.
Neighborhood — ranked by edge-count
Papers (1)
paper
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Variant of multi-head attention used in Llama3 and Qwen2.5 models evaluated in this paper
- Selective attention mechanism is a component of GWT-2 indicator; AST-1 models and enables control over attention
- Blocking attention edges to trace information flow.
- A predictive model representing and controlling attention; central to attention schema theory.
- Phenomenon where layers concentrate most attention weight on the first token, captured by ColSum Concentration metric
- Process using Q, K, V to compute a heat map over K and weighted sum of V.
- Ranks all heads by attention on a pair to trace composition.
- An Elephant action of answering a query.