concept
active
concept:attention-mechanism

attention mechanism

Core operation in transformers, computing weighted combinations of previous elements

Neighborhood — ranked by edge-count

Concepts (1)

concept

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

  • Process using Q, K, V to compute a heat map over K and weighted sum of V.
  • Attentionconcept0.841
    Selective attention mechanism is a component of GWT-2 indicator; AST-1 models and enables control over attention
  • Attention biologyconcept0.814
    Study of what attentional features and head behaviors exist.
  • Blocking attention edges to trace information flow.
  • A possible circuit that triggers when activations deviate from expected values, hypothesized to underlie noticing injected thoughts.
  • auction mechanismconcept0.796
    Competitive bidding to acquire animal cards, with buy-right option for auctioneer.
  • Attention Schemaconcept0.795
    A predictive model representing and controlling attention; central to attention schema theory.
  • attentional modeconcept0.793
    Variable aperture of attention, ranging from collapsed to expanded; human and model analogs.