concept
active
concept:attention-computation

attention computation

Process using Q, K, V to compute a heat map over K and weighted sum of V.

Neighborhood — ranked by edge-count

Concepts (1)

concept
  • Attention
    related_to
    Selective attention mechanism is a component of GWT-2 indicator; AST-1 models and enables control over attention

Artifacts (1)

artifact

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

  • Core operation in transformers, computing weighted combinations of previous elements
  • Attention streamsconcept0.837
    Term introduced by this paper for the horizontal axis of information flow in transformers across token-positions via attention heads
  • Attention biologyconcept0.826
    Study of what attentional features and head behaviors exist.
  • Attention Schemaconcept0.815
    A predictive model representing and controlling attention; central to attention schema theory.
  • Blocking attention edges to trace information flow.
  • Attentional features spread across multiple heads.
  • Attentional featuresframework0.807
    Proposed attention analog of sparse monosemantic features for decomposing attention layers.
  • Formal model of meditation phenomenology: focus → distraction → awareness of distraction → redirection, derived from active inference.