concept
active
concept:newline-attention-sink

Newline attention sink

Counting heads default attention to previous newline.

Neighborhood — ranked by edge-count

Concepts (1)

concept
  • Attention Sink
    related_to
    Phenomenon where layers concentrate most attention weight on the first token, captured by ColSum Concentration metric

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.