concept
active
concept:bottleneck-activationBottleneck Activation
A lower-dimensional activation that is the only pathway for information between higher-dimensional activations; e.g. the residual stream between MLP layers
Neighborhood — ranked by edge-count
Papers (1)
paper
Concepts (1)
concept
- Residual Streamassociated_withProposed pathway flowing through layers at each position; calculates K/V values that feed horizontal information flow.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Internal representations of the model on which probes operate; the method uses activations to rank datapoints.
- Compression-prediction trade-off; NIS encodes micro-states through an information bottleneck.
- Population structure mechanism implementing genetic assortment; cited as example of individuation mechanism in multicellularity.
- Large activation magnitudes in the residual stream that Queipo-de Llano et al. link to causing compression behavior and stages of inference
- Clamping activations along the Assistant Axis to remain above a minimum threshold (25th percentile), introduced as a stabilization method
- Failure mode for output-surjectivity: LLMs may lack capacity to predict all tokens due to rank constraints
- Standard method in mechanistic interpretability that intervenes on activations; VPD flips this paradigm by patching parameters.
- Limited capacity of the workspace relative to the sum of module capacities.