DeepSeek open-sourced its production AI engine — runs on Nvidia and Huawei chips

DeepSeek published DeepInfer, the inference engine it says runs its own production AI services, on GitHub on October 20 under the Apache-2.0 license, alongside a tokenizer tool called deepseek-recipe. The first release runs DeepSeek-V4.1-Flash in offline mode only, targeting two chip families: Nvidia's SM100 Blackwell GPUs and Huawei's Ascend 950 NPUs.

The engine bundles custom kernels, mixture-of-experts parallelism spread across chips, speculative decoding that guesses several tokens ahead before verifying them, and structured output support; DeepSeek credits the open-source vLLM engine as the base it was built on. Its own production figures for V4.1-Flash: one cluster serves 155 trillion tokens a day at a 96% input cache hit rate, with time-to-first-token around 400 milliseconds — DeepSeek's numbers, not independently tested.

It landed the same week Washington ordered DeepSeek scrubbed from every civilian agency computer: the lab putting its serving stack out in the open while the U.S. government treats its software as a supply-chain threat. Huawei's Ascend 950 sitting next to Nvidia as a first-class target is the tell — a top Chinese lab building for the chip it can actually buy.