concept
active
concept:attention-sink

Attention Sink

Phenomenon where layers concentrate most attention weight on the first token, captured by ColSum Concentration metric

Neighborhood — ranked by edge-count

Methods (1)

method
  • Observing tokens with abnormally high average activation to locate where R1 treats the response as starting.

Concepts (2)

concept

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.