method
active
method:stochastic-text-generation-next-token-predictionStochastic text generation (next token prediction)
The core mechanism of LLMs: predicting the next token based on previous context.
Neighborhood — ranked by edge-count
Concepts (1)
concept
- Inner monologue / chain-of-thought in LLMsassociated_withThe hidden reasoning steps generated by recent LLMs before visible output; mentioned in the technology section.
Related by similarity (8)
cosine ≥ 0.65 · no typed edgeEntities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.
- The training objective of LLMs: predicting the most likely next token given context; formally P(w_{n+1}|w_1...w_n)
- Training objective used for all neural network models in the paper; cross-entropy loss over predicted token sequences.
- Evidence humans and AI use different representations.
- LLMs generating text by feeding output back through context window; debated whether this constitutes algorithmic recurrence for RPT-1
- What is the right statistical model for explaining uncertainty dynamics in text generation?question0.739Central scientific question.
- Quantitative argument for the richness of quasi-psychological connections enabled by attention streams
- Derogatory term for LLMs; Nix's commentary frames it as camp opposite to nascent consciousness.
- Core interpretive claim providing mechanistic explanation for early persona formation