concept
active
concept:non-stationarity-of-lm-activationsNon-stationarity of LM activations
Time-dependent correlation structure in activations.
Neighborhood — ranked by edge-count
Papers (1)
paper
Methods (3)
method
- Autocorrelation analysisimplementsCosine similarity at lags to test stationarity.
- Projects token representation onto context subspace.
- Stationary surrogateaboutPermutation/diagonal-mean baseline.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Autocorrelation of LM activations varies with sequence position, unlike the stationary surrogate.finding0.782Fig. 2b,f.
- Mechanistic question AOs can't answer.
- Average number of nonzero feature entries per input; primary measure of activation sparsity in the autoencoder
- Activation probes perform similarly to LLM monitors at catching reward hacking at far lower costclaim0.739Practical claim.
- Emotion emerges early, peaks in middle layers, sharpens with scale, and persists across tokens in LLM activations per Zhang & Zhong 2025
- Core framing of AOs.
- Latent model activations when processing inputs framed from another agent's perspective
- Core slogan encapsulating the paradigm shift of VPD.