method
active
method:residual-stream-cv-injection

Residual Stream CV Injection

Technique of adding control vectors to model hidden states at mid-residual layers without weight updates

Neighborhood — ranked by edge-count

Methods (1)

method

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

  • Core activation intervention: add scaled vector to residual stream at layer l during completion
  • Residual Streamconcept0.820
    Proposed pathway flowing through layers at each position; calculates K/V values that feed horizontal information flow.
  • Technique to localize causally implicated hidden states by swapping residual stream activations between a true and false input and measuring downstream log-probability changes
  • The intermediate representations in transformer layers whose activations are patched and probed for truth information
  • The finite dimensional capacity of the residual stream for storing and communicating information between layers; conceptualized as being under high demand
  • Design choice to inject CVs at the middle residual layer where stylistic effects are strongest
  • Used to localize causally implicated hidden states by swapping activations between true and false inputs
  • The phenomenon where the residual stream communicates many more features than its dimensionality by encoding information across overlapping subspaces