concept
active
concept:flat-autoregressive-llmsflat autoregressive LLMs
Large language models without hierarchical structure, challenged by long sequences
Neighborhood — ranked by edge-count
Papers (1)
paper
Concepts (1)
concept
- long-range coherenceassociated_withAbility to maintain structural consistency over extended sequences
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- Second model system studied; used to show why flat autoregressive LLMs struggle with long-range coherence.
- The mechanism by which LLMs generate text: drawing a token from the next-token distribution and appending it to context repeatedly
- The core phenomenon studied: the ability of LLMs to evaluate and revise their own reasoning.
- Baseline persistence of any probe direction arising from the autoregressive nature of LLMs, not specific to emotion content
- Transformers are recurrent through autoregression because the K/V stream provides horizontal information flow across positions, even though each forward pass is feedforward.
- Alternative data attribution approach using an LLM as a judge; compared against the probe-based method.
- Statistical technique where outputs are regressed on previous values; used in language generation
- The central research question motivating the paper