hypothesis
active
hypothesis:with-an-attention-decomposition-the-line-width-head-s-three-behaviors-would-appear-as-three-separate-features-and-answer-selection-as-one-high-rank-featureWith an attention decomposition, the line width head's three behaviors would appear as three separate features and answer selection as one high-rank feature
Prediction for future decomposition.
Neighborhood — ranked by edge-count
Papers (1)
paper
Frameworks (1)
framework
- Attentional featuresaboutProposed attention analog of sparse monosemantic features for decomposing attention layers.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Line width head shows three behaviors: years split tokens, multi-token words, and newlinesfinding0.838Identified from attention patterns in Haiku 3.5.
- Author is fairly confident.
- Central theme.
- Theme across prior and own work.
- QK attribution for MC heads.
- Suspected but no confirmed example in real LLM.
- Does the answer selection head really implement one behavior, or many behaviors that share a common template?question0.782Raised by disjoint feature sets.
- Empirical observation from examining expanded OV/QK matrices; approximately 10 out of 12 heads show significant copying