finding
active
finding:line-width-head-shows-three-behaviors-years-split-tokens-multi-token-words-and-newlinesLine width head shows three behaviors: years split tokens, multi-token words, and newlines
Identified from attention patterns in Haiku 3.5.
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (2)
claim
- Author is fairly confident.
- Theme across prior and own work.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Prediction for future decomposition.
- Interesting special case of copying behavior related to tokenization artifacts; primitive precursor to induction heads
- Early-layer head counting characters since newline, found polysemantic.
- Constraint not surfaced by attributions.
- Steering planned-word features only affects the final token when applied at the newline planning token.finding0.745Planning is localized.
- A pair of query and key subcomponents distributed across attention heads performs previous-token behaviorfinding0.744VPD recovers an attention algorithm for attending to the previous token, distributed across multiple heads.
- Linear separability.
- Illustrative finding from Lindsey et al. 2025 / Hanna and Ameisen 2026 showing intention-like representations in attention streams