hypothesis
active
hypothesis:similar-superposition-phenomena-may-exist-in-self-attention-layers-and-similar-sparse-autoencoder-methods-may-extract-useful-structure-from-attentionSimilar superposition phenomena may exist in self-attention layers and similar sparse autoencoder methods may extract useful structure from attention
Extension of superposition hypothesis to attention layers as future research direction
Source paper
extracted_from(2024) · Marc Carauleanu · Michael Vaiana · Judd Rosenblatt · Cameron Berg +1
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Attentional features spread across multiple heads.
- Are heads mediating the same edge attending for the same reason, suggesting attention head superposition?question0.785Distributed functional units.
- Theoretical point.
- superposition is deeply linked to the ability to identify and enumerate over all features in a modelquote0.777Safety rationale.
- Supported by Table 1.
- Superposition maximalist stance.
- Interpretive claim about the mechanistic substrate of introspection in LLMs
- Structural finding about which attention heads control reflection behavior