concept
active
concept:attention-mechanismattention mechanism
Core operation in transformers, computing weighted combinations of previous elements
Neighborhood — ranked by edge-count
Concepts (1)
concept
- Selective attention mechanismrelated_toMechanism that selects information from modules for representation in the global workspace.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Process using Q, K, V to compute a heat map over K and weighted sum of V.
- Selective attention mechanism is a component of GWT-2 indicator; AST-1 models and enables control over attention
- Study of what attentional features and head behaviors exist.
- Blocking attention edges to trace information flow.
- A possible circuit that triggers when activations deviate from expected values, hypothesized to underlie noticing injected thoughts.
- Competitive bidding to acquire animal cards, with buy-right option for auctioneer.
- A predictive model representing and controlling attention; central to attention schema theory.
- Variable aperture of attention, ranging from collapsed to expanded; human and model analogs.