Next-Token Prediction (NTP)

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

• Next Token Predictionconcept0.882
  The training objective of LLMs: predicting the most likely next token given context; formally P(w_{n+1}|w_1...w_n)
• Stochastic text generation (next token prediction)method0.808
  The core mechanism of LLMs: predicting the next token based on previous context.
• Neutral Predictionconcept0.749
  NLI class indicating neither entailment nor contradiction; weighted 0 in Baseline but +1 in Neutral NLI Diversity
• Beyond NLPconcept0.712
  Extension of compositional methods beyond natural language processing to general AI and hardware
• In Llama 3 70B, each next-token prediction at token 101 draws on 64,000 independent attention streams (8 heads × 80 layers × 100 prior positions), each carrying a 128-dimensional signalfinding0.712
  Quantitative argument for the richness of quasi-psychological connections enabled by attention streams
• Neutral NLI predictions may capture lexical rather than semantic diversityhypothesis0.702
  Hypothesis proposed to explain Neutral NLI Diversity's high performance on decTest but low on conTest
• A pair of query and key subcomponents distributed across attention heads performs previous-token behaviorfinding0.701
  VPD recovers an attention algorithm for attending to the previous token, distributed across multiple heads.
• zero-shot predictionconcept0.700
  Prediction without task-specific training; Evee achieves 0.991 AUROC on indels in zero-shot mode.