method
active
method:autocorrelation-analysisAutocorrelation analysis
Cosine similarity at lags to test stationarity.
Neighborhood — ranked by edge-count
Papers (1)
paper
Concepts (1)
concept
- Non-stationarity of LM activationsimplementsTime-dependent correlation structure in activations.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Dot product of summed decoder vector with unembedding vector, following Hanna et al.
- The mechanism by which LLMs generate text: drawing a token from the next-token distribution and appending it to context repeatedly
- Second model system studied; used to show why flat autoregressive LLMs struggle with long-range coherence.
- Statistical technique where outputs are regressed on previous values; used in language generation
- Transformers are recurrent through autoregression because the K/V stream provides horizontal information flow across positions, even though each forward pass is feedforward.
- LLMs generating text by feeding output back through context window; debated whether this constitutes algorithmic recurrence for RPT-1
- Linear-model solution that trades off with superposition.
- Tuple retrieval by matching against a template, similar to relational database queries.