concept
active
concept:autoregressive-persistenceautoregressive persistence
Baseline persistence of any probe direction arising from the autoregressive nature of LLMs, not specific to emotion content
Neighborhood — ranked by edge-count
Methods (1)
method
- Controls for variance by sampling random directions from top-k PC spaces matching each emotion probe's explained variance, and subtracting median persistence of 20 matched directions
Concepts (2)
concept
- autoregressive recurrencerelated_toTransformers are recurrent through autoregression because the K/V stream provides horizontal information flow across positions, even though each forward pass is feedforward.
- residual persistenceextendsEmotion feature persistence above and beyond the persistence expected from high variance explained alone, computed by subtracting median variance-matched probe persistence
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Second model system studied; used to show why flat autoregressive LLMs struggle with long-range coherence.
- The mechanism by which LLMs generate text: drawing a token from the next-token distribution and appending it to context repeatedly
- The training parallelization technique that latent methods are difficult to train with.
- Statistical technique where outputs are regressed on previous values; used in language generation
- Training objective interpretable as optimizing a diverse set of tasks; thus subject to multitask scaling convergence pressures
- Autoregressive model unable to converge to a single stored pattern for any finite β (Corollary 2)finding0.787Consequence of Theorem 3 and 1D no-order result
- The phenomenon that emotion feature activations remain elevated above baseline beyond local token bursts, measurable as long-range correlation