claim
active
claim:the-residual-stream-has-a-deeply-linear-structure-enabling-virtual-weights-and-path-expansion-analysisThe residual stream has a deeply linear structure, enabling virtual weights and path expansion analysis
Architectural observation enabling the entire mathematical framework; the residual stream is purely a sum of linear projections
Neighborhood — ranked by edge-count
Concepts (2)
concept
- Virtual WeightssupportsImplicit weights directly connecting any pair of layers computed by multiplying output weights of one layer with input weights of another through the residual stream
- Path Expansion TricksupportsThe mathematical trick of expanding a product of layer terms into a sum of end-to-end path terms, enabling independent analysis of each term
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Proposed pathway flowing through layers at each position; calculates K/V values that feed horizontal information flow.
- The phenomenon where the residual stream communicates many more features than its dimensionality by encoding information across overlapping subspaces
- The middle layer residual stream features are causally implicated in multi-step reasoning.claim0.776Features for Kobe Bryant, California, Lakers participate in computing the capital answer.
- Third checkpointing strategy; adopted here.
- App. B.4.
- Core activation intervention: add scaled vector to residual stream at layer l during completion
- Supports monosemantic reading.
- Interpretive synthesis of DIM and cone intervention successes