
In January 2025, DeepSeek released R1 using RLVR (Reinforcement Learning with Verifiable Rewards) instead of the human-preference training pipeline. For math problems, the verifier checked if the model's answer matched the known solution; for code, a compiler ran the output and returned pass or fail (binary rewards: 1 for correct, 0 for wrong).
DeepSeek's approach uses GRPO (Group Relative Policy Optimization), which removes the critic model and learned reward model entirely. Instead of four full-size models in memory (policy, reference policy, reward model, critic), the system requires just two (the policy being trained and a reference copy for KL regularization), cutting memory demands substantially.
DeepSeek R1-Zero, trained with GRPO and verifiable rewards with no supervised fine-tuning, went from 15.6% to 77.9% on AIME 2024 math problems; with majority voting, it hit 86.7%, matching OpenAI's o1. The model developed self-verification, reflection, and chain-of-thought reasoning purely from the binary signal.
Summaries like this, every morning.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.