concept
active
concept:qk-circuitQK Circuit
The bilinear circuit formed by W_E^T W_QK^h W_E that determines which source token a destination token attends to
Neighborhood — ranked by edge-count
Papers (2)
paper
Concepts (1)
concept
- Induction Headsassociated_withMechanistic circuits in transformers documented by Olsson et al. 2022, cited as evidence for pattern-repository assumption
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- QK rotates character count manifold to align with line width at an offset.
- Induction QK circuits combine searching for any name and searching for names of aunts/unclesclaim0.750Interpretation of induction heads.
- The key novel property of DiffLogic CA — logic gate networks that are recurrent both spatially and temporally
- Scaling features inside QK circuits of selected heads to validate mechanisms.
- A circuit exhibiting symmetry such that the weights rotate with the orientation of the feature they detect
- A form of attention head composition where W_Q reads from a subspace affected by a previous head, allowing more complex attention patterns
- Anthropic technique revealing computational pathways in Claude.
- Across MCQ, sentiment, dialogue.