Proposition 4.2: Under bounded residual stream and weight-tied attention, the difference between attention weight matrices at successive recurrences is bounded by 2*L_sm*kappa_l*B*||Delta_{l,t}||

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Two-layer attention-only transformers implement much more complex algorithms via composition of attention heads, detectable directly from weightsclaim0.764
  Core claim for two-layer models; composition creates qualitatively more powerful in-context learning
• In the analyzed two-layer attention-only model, only K-composition is significant; V- and Q-composition are negligible by Frobenius norm measurefinding0.754
  Result from applying the Frobenius norm composition measurement to all attention head pairs in the two-layer model
• Some attention heads partially specialize in copying for words that split into two tokens without a space prefix, attending from fragmented token to complete tokenfinding0.752
  Interesting special case of copying behavior related to tokenization artifacts; primitive precursor to induction heads
• Birch's claim that only transcript and abstract weight similarity connect successive forward passes misses the richer quasi-psychological connections sustained by attention streamsclaim0.750
  Direct rebuttal of Birch 2025's skepticism about psychological continuity in LLMs
• The residual stream has a deeply linear structure, enabling virtual weights and path expansion analysisclaim0.748
  Architectural observation enabling the entire mathematical framework; the residual stream is purely a sum of linear projections
• Attention is a generalization of convolution; all convolutions can be expressed as tensor products of fixed relative position attention patterns and weight matricesclaim0.747
  Mathematical equivalence showing the relationship between attention mechanisms and convolutional operations
• Attention heads can be understood as independent operations each adding their output to the residual stream, equivalent to the concatenate-and-multiply formulationclaim0.746
  Mathematical equivalence enabling independent analysis of each attention head
• A pair of query and key subcomponents distributed across attention heads performs previous-token behaviorfinding0.746
  VPD recovers an attention algorithm for attending to the previous token, distributed across multiple heads.