DeepSeek-R1-Distill-Qwen-1.5B Surpasses GPT-4o in certain benchmarks

DeepSeek has introduced its first-generation reasoning models, DeepSeek-R1-Zero and DeepSeek-R1, which utilize large-scale reinforcement learning (RL) without prior supervised fine-tuning (SFT). DeepSeek-R1-Zero has shown impressive reasoning capabilities but faced issues like repetition and readability. To improve upon this, DeepSeek-R1 incorporates cold-start data before RL, achieving performance on par with OpenAI's models across various tasks. The models have been open-sourced, including several distilled versions based on Llama and Qwen architectures, with DeepSeek-R1-Distill-Qwen-32B setting new benchmarks. The development pipeline for DeepSeek-R1 includes two RL stages to enhance reasoning patterns and two SFT stages for foundational capabilities. The research demonstrates that larger model reasoning can be distilled into smaller models, which perform better than small models trained solely through RL. The evaluation results indicate that the distilled models excel in various benchmarks, and the open-source nature of these models aims to benefit the research community. Users can access the models via the DeepSeek platform and run them locally with specific configurations to avoid common issues like repetition.

- DeepSeek-R1 models utilize reinforcement learning without prior supervised fine-tuning.

- The models have been open-sourced, including several distilled versions.

- DeepSeek-R1-Distill-Qwen-32B has achieved new state-of-the-art results.

- The development pipeline includes stages for improving reasoning patterns and aligning with human preferences.

- Distilled models demonstrate superior performance compared to smaller models trained through RL alone.