concept
active
concept:superposition-in-residual-stream

Superposition in Residual Stream

The phenomenon where the residual stream communicates many more features than its dimensionality by encoding information across overlapping subspaces

Neighborhood — ranked by edge-count

Concepts (1)

concept
  • The finite dimensional capacity of the residual stream for storing and communicating information between layers; conceptualized as being under high demand

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.