concept
active
concept:bottleneck-activationBottleneck Activation
A lower-dimensional activation that is the only pathway for information between higher-dimensional activations; e.g. the residual stream between MLP layers
Neighborhood — ranked by edge-count
Papers (1)
paper
Concepts (1)
concept
- Residual Streamassociated_withProposed pathway flowing through layers at each position; calculates K/V values that feed horizontal information flow.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Internal representations of the model on which probes operate; the method uses activations to rank datapoints.
- Compression-prediction trade-off; NIS encodes micro-states through an information bottleneck.
- Population structure mechanism implementing genetic assortment; cited as example of individuation mechanism in multicellularity.
- Clamping activations along the Assistant Axis to remain above a minimum threshold (25th percentile), introduced as a stabilization method
- Failure mode for output-surjectivity: LLMs may lack capacity to predict all tokens due to rank constraints
- Standard method in mechanistic interpretability that intervenes on activations; VPD flips this paradigm by patching parameters.
- Limited capacity of the workspace relative to the sum of module capacities.
- The conventional approach (e.g., SAEs, transcoders) of decomposing activations into interpretable features.