finding
active
finding:removing-key-side-name-of-aunt-uncle-features-in-qk-circuits-eliminates-induction-in-haiku-which-predicts-generic-aunt-namesRemoving key-side 'name of aunt/uncle' features in QK circuits eliminates induction in Haiku, which predicts generic aunt names
Steering validation.
Source paper
extracted_fromNeighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Induction QK circuits combine searching for any name and searching for names of aunts/unclessupportsInterpretation of induction heads.
Methods (1)
method
- Scaling features inside QK circuits of selected heads to validate mechanisms.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- QK attribution of induction-relevant heads.
- QK attribution from Ivory to 'a' has top pair Ivory Coast query feature and Zambian key featurefinding0.719Explains attention edge.
- Perturbation validation.
- Steering validation for antonym mechanism.
- Across MCQ, sentiment, dialogue.
- The mechanistic explanation of how induction heads are implemented in two-layer models
- In the Aunt Mary Sue prompt, QK attributions matrix has rank at least 2 with no strong cross termsfinding0.701Parallel independent QK interactions.
- Fundamental design principle driving Oberon: proper abstraction enables modular structure and maintainability.