DeepEP over EFA lifts MoE RL rollout 40 percent on AWS

The post starts from a structural problem in post-training MoE models: as these architectures become more sparse to cut inference costs, training becomes limited more by communication between machines than by raw compute. Expert Parallelism worsens this, because it routes tokens dynamically across devices rather than using the structured patterns of tensor, data, or pipeline parallelism.

To address that, AWS combined Amazon EKS, EFA, and Amazon S3 so orchestration, high-performance networking, and durable storage can scale on their own. Amazon also contributed features migrating DeepEP's communication primitives to libfabric, giving DeepEP v2 native EFA support, and NCCL 2.31 picked up the latest EFA optimizations for dense collective communication. In testing across 48 P5en instances, that combination raised aggregate rollout throughput by 40 percent. The post also proposes Spot Instances for rollout generation, since interruption of a rollout worker does not halt the whole job the way it would for tightly coupled training workers.

Whether that 40 percent holds beyond this single cluster size is the practical uncertainty. For teams weighing where to spend on training infrastructure, the finding suggests networking configuration, not just accelerators, may be the lever worth testing first.