Massive Activations

Large activation magnitudes in the residual stream that Queipo-de Llano et al. link to causing compression behavior and stages of inference

Neighborhood — ranked by edge-count

paper

concept

Activations
related_to
Internal representations of the model on which probes operate; the method uses activations to rank datapoints.

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Activation Additionmethod0.768
Intervention method that adds a learned direction vector to residual stream activations to steer model behavior
Activation Oraclesframework0.759
Framework training LLMs to answer questions about externally-provided activation vectors
Other-Referencing Activationsconcept0.758
Latent model activations when processing inputs framed from another agent's perspective
Activation decompositionconcept0.754
The conventional approach (e.g., SAEs, transcoders) of decomposing activations into interpretable features.
Activation Probingconcept0.753
Technique of reading out model beliefs from internal activations before the final answer token is generated
Activation Cappingmethod0.752
Clamping activations along the Assistant Axis to remain above a minimum threshold (25th percentile), introduced as a stabilization method
Activation Compressionconcept0.752
Key capability: covariance pooling compresses gigabytes of activations into compact stable embeddings without large labeled datasets.
Bottleneck Activationconcept0.752
A lower-dimensional activation that is the only pathway for information between higher-dimensional activations; e.g. the residual stream between MLP layers