$38 Per GPU Hour: DeepSeek Open-Sources DeepInfer and Shows Its Inference Ledger

On October 20, DeepSeek announced the full open-sourcing of its internal inference framework DeepInfer on GitHub — and, unusually, published its production ledger: 155 trillion tokens of daily throughput per cluster and $38 per GPU-hour, implying over $27,000 of monthly revenue per card. The "black box" of LLM inference economics has been pried open.

Open-sourcing the engine, not just the model

Per Jiemian and Cailian Press, DeepInfer is the internal framework behind V4.1 Flash's high-throughput, low-latency serving at scale; future models' inference code and optimizations will land in the same repository. A production-grade token codec component, deepseek-recipe, was released alongside. DeepSeek says it wants to help developers and enterprises optimize throughput and latency, and build private- and public-cloud deployments efficiently.

The ledger's significance exceeds its numbers: $38/GPU-hour implies roughly $27,360 per card per month under full load — a benchmark of unit economics for enterprises weighing self-built clusters against API rentals. Ascend compatibility widens the chip options for private deployment.

Whose cheese does an open inference framework move?

The stated goals: cut compute cost and energy use so "intelligence reaches more people at lower prices." That puts DeepInfer in direct competition with vLLM, SGLang and other open stacks — and forces every cloud vendor to re-answer whether inference should stay in-house. Related compute and AI-application stocks rallied on the news.

Open models set the ceiling of intelligence; open inference frameworks set its unit price.

NineZenith's own compute-scheduling practice follows the same logic: throughput, latency and unit cost must be measurable, optimizable engineering metrics before industry AI can scale. We are evaluating DeepInfer's fit on our own compute fabric.

(Compiled from public reports by Jiemian, Cailian Press, Wallstreetcn, Yicai and Observer)