Key-query attention

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Attention Schemaconcept0.745
  A predictive model representing and controlling attention; central to attention schema theory.
• attention computationconcept0.739
  Process using Q, K, V to compute a heat map over K and weighted sum of V.
• answer.querymethod0.733
  An Elephant action of answering a query.
• Position-Only Keys/Queries, Stimulus-Only Values Factorizationmethod0.732
  Key architectural modification restricting queries and keys to position encodings while values depend only on stimuli; extreme version of best-practice insight.
• A pair of query and key subcomponents distributed across attention heads performs previous-token behaviorfinding0.727
  VPD recovers an attention algorithm for attending to the previous token, distributed across multiple heads.
• lock-and-key relationshipframework0.726
  The highly specific fit between a new building volume and the existing configuration, analogous to protein-ligand binding.
• Attention is All You Need (Vaswani et al., 2017)concept0.718
  Original transformer paper; foundational reference cited throughout for the architecture being analyzed.
• Key, query, and value vectors are intermediary byproducts; W_OV and W_QK are the fundamental low-rank matrices describing attention head behaviorclaim0.713
  Reframing observation: the canonical K/Q/V decomposition is computationally convenient but not the most interpretable representation