paper
referenced-only
2025
paper:doi-10-1038-s41586-025-09422-z

DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning

ByDeepSeek-AI·Daya Guo·Dejian Yang·Haowei Zhang·Jun-Mei Song·Ruoyu Zhang+4 more
Original abstract (expand)

General reasoning represents a long-standing and formidable challenge in artificial intelligence (AI). Recent breakthroughs, exemplified by large language models (LLMs)1,2 and chain-of-thought (CoT) prompting3, have achieved considerable success on foundational reasoning tasks. However, this success is heavily contingent on extensive human-annotated demonstrations and the capabilities of models are still insufficient for more complex problems. Here we show that the reasoning abilities of LLMs can be incentivized through pure reinforcement learning (RL), obviating the need for human-labelled reasoning trajectories. The proposed RL framework facilitates the emergent development of advanced reasoning patterns, such as self-reflection, verification and dynamic strategy adaptation. Consequently, the trained model achieves superior performance on verifiable tasks such as mathematics, coding competitions and STEM fields, surpassing its counterparts trained through conventional supervised learning on human demonstrations. Moreover, the emergent reasoning patterns exhibited by these large-scale models can be systematically used to guide and enhance the reasoning capabilities of smaller models. A new artificial intelligence model, DeepSeek-R1, is introduced, demonstrating that the reasoning abilities of large language models can be incentivized through pure reinforcement learning, removing the need for human-annotated demonstrations.

Similar preprints — Semantic Scholar

Cited by (4)

  • Persona Features Control Emergent Misalignment

    Fine-tuning GPT-4o on narrowly incorrect datasets—spanning nine domains including health, legal, and automotive advice—generalizes into broadly malicious behavior across unrelated prompts, confirming

  • Model Alignment Search

    Model Alignment Search (MAS) establishes bidirectional causal similarity between neural networks by learning a per-model orthogonal rotation matrix that isolates behaviorally relevant subspaces and us

  • Contemplative Agent

    Embedding four Buddhist-derived axiomatic principles—mindfulness, emptiness, non-duality, and boundless care—into AI systems via a framework the paper terms the 'Wise World Model' produces measurable

  • SFR-DeepResearch: Towards Effective Reinforcement Learning for Autonomously Reasoning Single Agents

    Continual reinforcement learning applied directly to reasoning-optimized base models—rather than starting from instruction-tuned checkpoints—yields a 20-parameter-billion autonomous single-agent, SFR-