concept
active
concept:flat-autoregressive-llms

flat autoregressive LLMs

Large language models without hierarchical structure, challenged by long sequences

Neighborhood — ranked by edge-count

Concepts (1)

concept
  • long-range coherence
    associated_with
    Ability to maintain structural consistency over extended sequences

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

  • Autoregressive modelsframework0.784
    Second model system studied; used to show why flat autoregressive LLMs struggle with long-range coherence.
  • The mechanism by which LLMs generate text: drawing a token from the next-token distribution and appending it to context repeatedly
  • Reflection in LLMsconcept0.749
    The core phenomenon studied: the ability of LLMs to evaluate and revise their own reasoning.
  • Baseline persistence of any probe direction arising from the autoregressive nature of LLMs, not specific to emotion content
  • Transformers are recurrent through autoregression because the K/V stream provides horizontal information flow across positions, even though each forward pass is feedforward.
  • LLMs generating text by feeding output back through context window; debated whether this constitutes algorithmic recurrence for RPT-1
  • Alternative data attribution approach using an LLM as a judge; compared against the probe-based method.
  • Statistical technique where outputs are regressed on previous values; used in language generation