concept
active
concept:context-windowcontext window
Finite number of previous tokens used by autoregressive models to predict the next token; defines interaction range
Neighborhood — ranked by edge-count
Frameworks (1)
framework
- AR(ω) modelassociated_withStochastic process model predicting next token from a context window of length ω; mapped to local Hamiltonian
Concepts (2)
concept
- Context Featurerelated_toFeature that activates across all tokens within a specific context (e.g., DNA sequences, base64 strings)
- causal maskingimplementsAttention restricted to previous tokens only, as in decoder-only models; leads to AR(ω)-like behaviour and no ordered phase
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- The process by which a form responds to its unique local conditions, leading to uniqueness.
- Basic data type in FCA consisting of a triple (G, M, I) representing objects, attributes, and incidence relations.
- Deep responsiveness to local conditions, essential for a process to be living.
- Feature that fires on a specific token only within a specific surrounding context (e.g., 'the' in physics vs 'the' in mathematics)
- Model outputs influenced by information from training documents not present in context; relevant to synthetic document fine-tuning results
- Symmetries of a formal context that can be exploited to reduce computation and derive general rules.
- A window that functions as a three-dimensional center, intensifying the life of the room and providing a comfortable spot by the light.