DeepSeek open-sources DeepInfer at $38 per accelerator-hour

DeepInfer descends from vLLM, which DeepSeek customized and released under Apache 2.0, adding components on top of work from SGLang and TRRTLLM. The open-source lineage is explicit: vLLM thanked DeepSeek for the release and credit, while Woosuk Kwon described DeepSeek as an early adopter and contributor. DeepInfer is therefore better understood as production learnings flowing back into an existing ecosystem than as an entirely separate replacement for it.

That lineage matters because vLLM supplied the most concrete comparative response. After acknowledging that DeepInfer was initially built primarily on vLLM, the vLLM team reported decode throughput within 6% of DeepInfer's published results on GB200 under a matched configuration, and published AgentX results and reproduction recipes. That materially qualifies claims that DeepSeek's efficiency is inaccessible to existing open-source engines.

The release also triggered a second debate: whether the disclosures demonstrate high inference profitability more generally. A widely shared estimate initially put inference margins at 87.76–96.94%, before its author corrected it to 84.2–96.1% and noted that the idealized scenario implied roughly $49/hour against DeepSeek's reported $38/hour. Other posts use higher hardware costs, spanning approximately 73.7–86.8% at $5–$10/hour. Those calculations omit or incompletely allocate the SSD cache fleet, CPUs, networking, idle capacity, failures, engineering and support costs, and they depend on a highly favorable cache-heavy token mix, so high contribution margins on inference can coexist with large company-level losses.