concept
active
concept:v-compositionV-Composition
A form of attention head composition where W_V reads from a subspace affected by a previous head, creating virtual attention heads
Neighborhood — ranked by edge-count
Thinkers (1)
thinker
- Neel NandastudiesExternal commenter; resolved apparent counterexample to linear representation hypothesis
Concepts (2)
concept
- K-Compositionassociated_withA form of attention head composition where W_K reads from a subspace affected by a previous head; central to how induction heads are implemented
- Virtual Attention HeadimplementsThe composition of two attention heads via V-composition, forming a new entity with its own attention pattern A^h2 * A^h1 and OV matrix W_OV^h2 * W_OV^h1
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- The wiring together of processes to form new processes in process theory
- A form of attention head composition where W_Q reads from a subspace affected by a previous head, allowing more complex attention patterns
- Publication series in which the work appeared.
- Modeling function application via feedback loops between processes, ping-ponging tokens.
- Composition where a whole cannot be meaningfully decomposed into its original parts, central to Schrödinger compositional theory
- In attention, value vectors that carry the information future positions should receive.
- Denotation function µ decomposes over operations so meaning of compound expressions follows from meanings of parts
- Central concept: how meaning of wholes depends on meanings of parts and their structural arrangement; multiple formulations explored (Frege, Schrödinger, Whitehead, LEGO).