Attention Sink Score

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Attention Sinkconcept0.885
  Phenomenon where layers concentrate most attention weight on the first token, captured by ColSum Concentration metric
• Attention streamsconcept0.758
  Term introduced by this paper for the horizontal axis of information flow in transformers across token-positions via attention heads
• Sink Ratemethod0.753
  Fraction of attention heads with sink score above threshold 0.3, used to track stages of inference
• attention computationconcept0.744
  Process using Q, K, V to compute a heat map over K and weighted sum of V.
• Importance Scoringmethod0.734
  Weighted Spearman correlation that corrects for sampling bias in automated interpretability evaluation
• Probe scoreconcept0.733
  Dot product between hidden state and concept vector averaged across 5-layer window around best layer; measures model's internal emotive state
• Attention Decayconcept0.729
  Decrease in attention paid to system prompt over conversational turns, leading to persona fidelity degradation (cited from Li et al.)
• Attentionconcept0.727
  Selective attention mechanism is a component of GWT-2 indicator; AST-1 models and enables control over attention