concept
active
concept:attention-biology

Attention biology

Study of what attentional features and head behaviors exist.

Neighborhood — ranked by edge-count

Concepts (4)

concept

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

  • Process using Q, K, V to compute a heat map over K and weighted sum of V.
  • Core operation in transformers, computing weighted combinations of previous elements
  • Attention streamsconcept0.811
    Term introduced by this paper for the horizontal axis of information flow in transformers across token-positions via attention heads
  • Attention headsconcept0.793
    Transformer attention heads that could be recruited to extract different kinds of information (text vs. thoughts).
  • Self-attentionconcept0.792
    A form of key-query attention within a single input sequence; core to Transformers.
  • Analytical Biologyframework0.791
    Sommerhoff's 1950 work on teleology and purposive behavior in organisms; relevant to biological self-regulation.
  • Attention Schemaconcept0.785
    A predictive model representing and controlling attention; central to attention schema theory.
  • Blocking attention edges to trace information flow.