concept
active
concept:qk-twistingQK twisting
QK rotates character count manifold to align with line width at an offset.
Neighborhood — ranked by edge-count
Papers (1)
paper
- When Models Manipulate Manifolds: The Geometry of a Counting Taskintroducesmentions
Concepts (1)
concept
- Boundary headsimplementsAttention heads whose QK twists count manifolds to detect approaching line end.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- The bilinear circuit formed by W_E^T W_QK^h W_E that determines which source token a destination token attends to
- Core method decomposing attention scores as bilinear sums of query-key feature pairs.
- Scaling features inside QK circuits of selected heads to validate mechanisms.
- Writes attention scores/head output as sums of feature interactions; core HeadVis capability.
- Future work.
- Model-free RL algorithm used in experimental comparison; employs ε-greedy exploration.
- QK attribution from Ivory to 'a' has top pair Ivory Coast query feature and Zambian key featurefinding0.668Explains attention edge.
- Rotation about a point by an angle 360°/n leaving the pattern invariant