question
active
question:will-a-head-perform-the-same-algorithm-across-prompts-or-are-individual-heads-polysemanticWill a head perform 'the same algorithm' across prompts, or are individual heads polysemantic?
Head polysemanticity.
Source paper
extracted_fromNeighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Attentional circuits employ many computations and heuristics in parallel, even in simple contextsgatesNew mechanism insight.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Does the answer selection head really implement one behavior, or many behaviors that share a common template?question0.810Raised by disjoint feature sets.
- A head can implement multiple behaviors via high-rank output without superposition.
- Best guess but not fully convinced; lacks crisp text description to test.
- Claim supported by VPD's recovery of cross-head attention subcomponents, noted in footnote.
- Central theme.
- Theme across prior and own work.
- Theoretical point.
- Motivation.