concept
active
concept:self-attentionSelf-attention
A form of key-query attention within a single input sequence; core to Transformers.
Neighborhood — ranked by edge-count
Papers (1)
paper
Frameworks (2)
framework
- TEM-Transformer (TEM-t)implementsThe transformer version directly analogous to TEM, introduced in this paper, offering dramatic performance improvements.
- Hopfield Networkanalogous_toA recurrent connectionist architecture that implements associative memory and distributed computation through symmetric weighted connections and Hebbian learning rules. The network converges to stable states through recurrent dynamics, enabling both memory retrieval and combinatorial problem-solving in a fully distributed manner.
Concepts (2)
concept
- Self Awarenessrelated_to
- Paper showing Hopfield networks are closely related to transformers; key intermediary result used to connect TEM to transformers.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- A dialogue agent using first-personal pronouns and expressing self-concern in ways that suggest consciousness but are actually role play
- The ability of a model to observe its own state, measured by Koan Battery; can be lifted by contemplative prompts.
- Process of reifying one's identity as an independent self; meditation practices aim to decrease selfing.
- Transformer attention heads that could be recruited to extract different kinds of information (text vs. thoughts).
- The ability of reasoning LLMs to review and revise previous reasoning steps during inference
- The model's verbal description of its internal state, which may be accurate or confabulated.
- The interior awareness, consciousness, and felt identity that each person experiences; absent from mechanistic cosmology.
- Spontaneous emergence of long-range order in networks; modeled as neural and basal cognition.