concept
active
concept:key-query-attention

Key-query attention

Attention mechanism in AI used in Transformers; also proposed for GWT implementations.

Neighborhood — ranked by edge-count

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

  • Variant of multi-head attention used in Llama3 and Qwen2.5 models evaluated in this paper
  • Attentionconcept0.769
    Selective attention mechanism is a component of GWT-2 indicator; AST-1 models and enables control over attention
  • Blocking attention edges to trace information flow.
  • Attention Schemaconcept0.745
    A predictive model representing and controlling attention; central to attention schema theory.
  • Attention Sinkconcept0.741
    Phenomenon where layers concentrate most attention weight on the first token, captured by ColSum Concentration metric
  • Process using Q, K, V to compute a heat map over K and weighted sum of V.
  • Ranks all heads by attention on a pair to trace composition.
  • answer.querymethod0.733
    An Elephant action of answering a query.