Superposition in Residual Stream

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Residual Streamconcept0.845
  Proposed pathway flowing through layers at each position; calculates K/V values that feed horizontal information flow.
• Superposition of Simulacraconcept0.798
  The state in which a dialogue agent maintains multiple possible characters simultaneously, refined as the conversation proceeds
• Residual Stream Patchingmethod0.796
  Technique to localize causally implicated hidden states by swapping residual stream activations between a true and false input and measuring downstream log-probability changes
• Residual-Stream Injectionconcept0.794
  Core activation intervention: add scaled vector to residual stream at layer l during completion
• Superpositionconcept0.793
  Phenomenon where models represent more features than dimensions via almost-orthogonal directions.
• Residual Stream Activationconcept0.790
  The intermediate representations in transformer layers whose activations are patched and probed for truth information
• The residual stream has a deeply linear structure, enabling virtual weights and path expansion analysisclaim0.782
  Architectural observation enabling the entire mathematical framework; the residual stream is purely a sum of linear projections
• Superposition in Neural Networksconcept0.779
  Theoretical model of how neural networks encode more features than dimensions, informing linear representation work.