residual stream recovery dynamics

The network's tendency to actively attenuate injected perturbations over subsequent layers, erasing the signal before output

Neighborhood — ranked by edge-count

framework

Computational Account of Layer-Dependent Introspection
supports
This paper's proposed mechanistic explanation integrating signal injection, attention routing, predictive integration, and residual recovery

claim

method

residual stream recovery tracking
implements
Tracks cosine similarity, norm ratio, and injection direction projection across layers to measure recovery from perturbation

concept

predictive integration
associated_with
The mid-to-late layer computational process that converts routed perturbation signals into explicit predictions

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Residual Streamconcept0.839
Proposed pathway flowing through layers at each position; calculates K/V values that feed horizontal information flow.
Residual Stream Activationconcept0.803
The intermediate representations in transformer layers whose activations are patched and probed for truth information
Residual Stream Patchingmethod0.801
Technique to localize causally implicated hidden states by swapping residual stream activations between a true and false input and measuring downstream log-probability changes
Residual Stream Bandwidthconcept0.787
The finite dimensional capacity of the residual stream for storing and communicating information between layers; conceptualized as being under high demand
Residual-Stream Injectionconcept0.777
Core activation intervention: add scaled vector to residual stream at layer l during completion
Residual Stream Activation Patchingmethod0.776
Used to localize causally implicated hidden states by swapping activations between true and false inputs
Superposition in Residual Streamconcept0.760
The phenomenon where the residual stream communicates many more features than its dimensionality by encoding information across overlapping subspaces
The residual stream has a deeply linear structure, enabling virtual weights and path expansion analysisclaim0.740
Architectural observation enabling the entire mathematical framework; the residual stream is purely a sum of linear projections