method
active
method:freezing-attention-patterns-trick

Freezing Attention Patterns Trick

A conceptual technique of fixing attention patterns to make the transformer a purely linear function of tokens, enabling independent analysis of OV and QK circuits

Neighborhood — ranked by edge-count

Concepts (1)

concept
  • The circuit formed by W_U W_OV^h W_E that describes how a given token affects output logits if attended to

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

  • Process using Q, K, V to compute a heat map over K and weighted sum of V.
  • Blocking attention edges to trace information flow.
  • Used to backpropagate through random masks.
  • Attention patterns scaled by the norm of the value vector at each source position, showing how large a vector is moved from each position
  • Attentional features spread across multiple heads.
  • Modification to transformer restricting keys and values to previous time-steps only, mimicking how an agent accumulates experiences.
  • Attention streamsconcept0.705
    Term introduced by this paper for the horizontal axis of information flow in transformers across token-positions via attention heads
  • Visualizing attention patterns weighted by the norm of value vectors to better show how much information is moved from each position