DORA system achieves 2–4× speedup in reinforcement learning training for large-scale language models by eliminating bottlenecks in generation phase

Researchers introduced DORA (Dynamic ORchestration for Asynchronous Rollout), an algorithm-system co-design that uses multi-version streaming rollout to overlap text generation with training, addressing the long-tailed trajectory problem that accounts for 50–80% of total training step time.

DORA maintains multiple policy versions (versions of the model weights) concurrently while preserving three convergence constraints: intra-trajectory policy consistency, data integrity, and bounded staleness—allowing full bubble elimination (removing idle waiting periods) without compromising algorithmic correctness.

In experiments, DORA achieved throughput improvements of up to 2–3 times higher than state-of-the-art systems on open-source benchmarks, and in large-scale industrial applications with tens of thousands of accelerators, accelerated RL training by 2–4 times compared to synchronous training across various scenarios.

The resulting open-source models, LongCat-Flash-Thinking, exhibit competitive performance on complex reasoning benchmarks, matching the capability of most advanced LLMs.