concept
active
concept:same-set-suppressionSame-set suppression
Head suppresses attention between related items, e.g., country and its own cities.
Neighborhood — ranked by edge-count
Papers (1)
paper
- HeadVis: An Interactive Tool For Investigating Attention Headsintroducesmentions
Concepts (1)
concept
- K-Compositionassociated_withA form of attention head composition where W_K reads from a subspace affected by a previous head; central to how induction heads are implemented
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Supported by geometry and 98% result but not a full account.
- Neural phenomenon reproduced by active inference model: reduced response to repeated stimuli.
- Behavior of IOI inhibition head and GPT-2 L10H7.
- Maintenance of tissue-level order; disruption of bioelectric networks can lead to tumorigenesis, which can be reversed by electrical normalization.
- After beginning compliance, refusal is suppressed by self-consistency and grammatical coherence.claim0.706Jailbreak persistence.
- Mechanism by which activation of an emotion feature sometimes leads to later suppression of that same featurequestion0.705Identified research gap: the paper observes anti-persistence but has no explanation for it
- Information-theoretic concept used to formalize why representative distributions are diverse; sequences in the typical set exhibit empirical statistics consistent with the pretraining distribution
- The idea that a shared structural language makes individual variations more apparent and lovable.