concept
active
concept:context-window

context window

Finite number of previous tokens used by autoregressive models to predict the next token; defines interaction range

Neighborhood — ranked by edge-count

Frameworks (1)

framework
  • AR(ω) model
    associated_with
    Stochastic process model predicting next token from a context window of length ω; mapped to local Hamiltonian

Concepts (2)

concept
  • Context Feature
    related_to
    Feature that activates across all tokens within a specific context (e.g., DNA sequences, base64 strings)
  • causal masking
    implements
    Attention restricted to previous tokens only, as in decoder-only models; leads to AR(ω)-like behaviour and no ordered phase

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

  • The process by which a form responds to its unique local conditions, leading to uniqueness.
  • Formal Contextconcept0.776
    Basic data type in FCA consisting of a triple (G, M, I) representing objects, attributes, and incidence relations.
  • Deep responsiveness to local conditions, essential for a process to be living.
  • Feature that fires on a specific token only within a specific surrounding context (e.g., 'the' in physics vs 'the' in mathematics)
  • Model outputs influenced by information from training documents not present in context; relevant to synthetic document fine-tuning results
  • Symmetries of a formal context that can be exploited to reduce computation and derive general rules.
  • Window Placeconcept0.728
    A window that functions as a three-dimensional center, intensifying the life of the room and providing a comfortable spot by the light.