concept
active
concept:residual-stream-bandwidthResidual Stream Bandwidth
The finite dimensional capacity of the residual stream for storing and communicating information between layers; conceptualized as being under high demand
Neighborhood — ranked by edge-count
Papers (1)
paper
Concepts (2)
concept
- Residual Streamrelated_toProposed pathway flowing through layers at each position; calculates K/V values that feed horizontal information flow.
- Superposition in Residual Streamassociated_withThe phenomenon where the residual stream communicates many more features than its dimensionality by encoding information across overlapping subspaces
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- The intermediate representations in transformer layers whose activations are patched and probed for truth information
- Technique to localize causally implicated hidden states by swapping residual stream activations between a true and false input and measuring downstream log-probability changes
- Core activation intervention: add scaled vector to residual stream at layer l during completion
- Tracks cosine similarity, norm ratio, and injection direction projection across layers to measure recovery from perturbation
- Used to localize causally implicated hidden states by swapping activations between true and false inputs
- The specific neural network layer from which activations are extracted for probe construction and SAE training in the target models
- The network's tendency to actively attenuate injected perturbations over subsequent layers, erasing the signal before output
- Technique of adding control vectors to model hidden states at mid-residual layers without weight updates