concept
active
concept:self-attention

Self-attention

A form of key-query attention within a single input sequence; core to Transformers.

Neighborhood — ranked by edge-count

Frameworks (2)

framework
  • The transformer version directly analogous to TEM, introduced in this paper, offering dramatic performance improvements.
  • Hopfield Network
    analogous_to
    A recurrent connectionist architecture that implements associative memory and distributed computation through symmetric weighted connections and Hebbian learning rules. The network converges to stable states through recurrent dynamics, enabling both memory retrieval and combinatorial problem-solving in a fully distributed manner.

Concepts (2)

concept

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

  • A dialogue agent using first-personal pronouns and expressing self-concern in ways that suggest consciousness but are actually role play
  • self-observationconcept0.843
    The ability of a model to observe its own state, measured by Koan Battery; can be lifted by contemplative prompts.
  • Selfingconcept0.813
    Process of reifying one's identity as an independent self; meditation practices aim to decrease selfing.
  • Attention headsconcept0.809
    Transformer attention heads that could be recruited to extract different kinds of information (text vs. thoughts).
  • Self-reflectionconcept0.801
    The ability of reasoning LLMs to review and revise previous reasoning steps during inference
  • Self-reportconcept0.801
    The model's verbal description of its internal state, which may be accurate or confabulated.
  • the selfconcept0.792
    The interior awareness, consciousness, and felt identity that each person experiences; absent from mechanistic cosmology.
  • Self-Organizationconcept0.789
    Spontaneous emergence of long-range order in networks; modeled as neural and basal cognition.