question
active
question:is-softmax-competition-for-priority-logic-a-fundamental-part-of-the-attention-mechanismIs softmax competition for priority logic a fundamental part of the attention mechanism?
Open question about competition among keys.
Neighborhood — ranked by edge-count
Papers (1)
paper
Concepts (1)
concept
- Heads attend to A if present else B via key scoring.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Interpretation of neural implementation in Section 5.1.
- Neural plausibility argument for softmax policy selection.
- Claim supported by VPD's recovery of cross-head attention subcomponents, noted in footnote.
- Attentional circuits employ many computations and heuristics in parallel, even in simple contextsclaim0.748New mechanism insight.
- Mathematical equivalence showing the relationship between attention mechanisms and convolutional operations
- Mechanism that selects information from modules for representation in the global workspace.
- Core operation in transformers, computing weighted combinations of previous elements
- Empirical observation from examining expanded OV/QK matrices; approximately 10 out of 12 heads show significant copying