concept
active
concept:attention-biologyAttention biology
Study of what attentional features and head behaviors exist.
Neighborhood — ranked by edge-count
Papers (1)
paper
Concepts (4)
concept
- Attentionrelated_toSelective attention mechanism is a component of GWT-2 indicator; AST-1 models and enables control over attention
- Attention superpositionassociated_withAttentional features spread across multiple heads.
- Head polysemanticity without superpositionassociated_withA head can implement multiple behaviors via high-rank output without superposition.
- High-rank attentional featuresassociated_withFeatures whose QK/OV are high rank, hard to interpret.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Process using Q, K, V to compute a heat map over K and weighted sum of V.
- Core operation in transformers, computing weighted combinations of previous elements
- Term introduced by this paper for the horizontal axis of information flow in transformers across token-positions via attention heads
- Transformer attention heads that could be recruited to extract different kinds of information (text vs. thoughts).
- A form of key-query attention within a single input sequence; core to Transformers.
- Sommerhoff's 1950 work on teleology and purposive behavior in organisms; relevant to biological self-regulation.
- A predictive model representing and controlling attention; central to attention schema theory.
- Blocking attention edges to trace information flow.