claim
active
claim:an-attention-head-can-be-polysemantic-without-superposition-because-its-output-is-high-rank-unlike-an-mlp-neuronAn attention head can be polysemantic without superposition because its output is high-rank, unlike an MLP neuron
Theoretical point.
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (1)
claim
- Author is fairly confident.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- What are the specific attention heads or MLP neurons (circuits) responsible for self-reflection in LLMs?question0.805Future research question about pinpointing fine-grained mechanistic components of reflection.
- Are heads mediating the same edge attending for the same reason, suggesting attention head superposition?question0.803Distributed functional units.
- Structural finding about which attention heads control reflection behavior
- Superposition hypothesis: neural networks represent more features than dimensions using almost-orthogonal directions.hypothesis0.798Explanation for why dictionary learning can recover many more features than dimensions.
- Precise characterization of why polysemanticity poses a combinatorial obstacle to circuit analysis
- Hypothesis based on observed negative cosine similarity between input and output weights of some neurons
- Architectural rationale for why Style Modulation Heads are in attention layers rather than MLPs
- Will a head perform 'the same algorithm' across prompts, or are individual heads polysemantic?question0.785Head polysemanticity.