finding
active
finding:autocorrelation-of-lm-activations-varies-with-sequence-position-unlike-the-stationary-surrogateAutocorrelation of LM activations varies with sequence position, unlike the stationary surrogate.
Fig. 2b,f.
Source paper
extracted_from(2025) · Lubana, Ekdeep Singh · Rager, Can · Hindupur, Sai Sumedh R. · Costa, Valerie +12
Neighborhood — ranked by edge-count
Papers (1)
paper
Claims (2)
claim
- SAE priors are in direct conflict with the non-stationary temporal structure of LM activations.supportsMain thesis.
- Proposition 4.1 central claim.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Fig. 2a,e.
- Time-dependent correlation structure in activations.
- Claim that capability emerges from architecture, not data, and that later models lose the surprise.
- Main result from Experiment 3 on effect of fine-tuning on HH-intent.
- Demonstrates bidirectional causal link: behavior manifold geometry can be recovered by optimizing in representation space.
- Limitation acknowledgment about the adequacy of the linear representation assumption
- Activation alignment algorithm succeeded on 99.2% of Llama/Qwen and 99.1% of GPT-OSS/Deepseek sequences.finding0.753Alignment failure analysis.
- If the relationship is present in activations, better tools (e.g. trained on these datasets) could surface ithypothesis0.752Alternative explanation.