claim
active
claim:attention-scores-are-a-bilinear-function-of-residual-stream-at-query-and-key-positions-so-can-be-written-as-sums-of-feature-feature-dot-products

Attention scores are a bilinear function of residual stream at query and key positions, so can be written as sums of feature-feature dot products

Core insight.

Neighborhood — ranked by edge-count

Methods (1)

method
  • Core method decomposing attention scores as bilinear sums of query-key feature pairs.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.