01What Is DeepSeek DeepInfer and How Does It Work?
DeepSeek DeepInfer is the open-source inference engine that DeepSeek uses to run its own AI for millions of users. On October 20, 2026, DeepSeek released it for free. An inference engine is the software that takes a user's question, splits the work across computer chips, writes the answer, and does it without making people wait.
Without the engine, a model is just a set of weights — like a recipe without a kitchen. DeepSeek DeepInfer is that kitchen.
Before this release, DeepSeek only shared small pieces of its engine, like FlashMLA and DeepGEMM. Now the whole system is open. DeepSeek said it wants to help everyone lower compute costs, use less energy, and make AI more affordable.
The engine handles 155 trillion tokens per day on DeepSeek's own service. That number shows how much work it can handle.
02Main Features of DeepSeek DeepInfer
DeepSeek DeepInfer includes several features that directly affect speed and cost. These are the ones the episode focused on:
- Speculative decoding (DSpark): A small helper model guesses the next few tokens. The big model checks all guesses at once. DSpark also has a confidence head that predicts which guesses will be accepted, cutting wasted checks by about 30%. DeepSeek estimates a 4% to 7% throughput gain at production load.
- Prefill shortcut: Before the AI answers, it must read the whole question. DeepSeek found it only needs to re-run the last 128 tokens of each prompt through certain layers. This nearly halves the total prefill work. For agents sending long prompts, this is a huge cost saver.
- Structured outputs: The AI can be forced to answer in a set format, like a form. This is vital for agents that pass data to other tools.
- Deterministic sampling: Ask the same question under the same settings, and you get the same answer every time. This makes testing reliable. These features only work on NVIDIA chips for now.
- Chip support: Runs on NVIDIA SM100 (Blackwell) and Huawei Ascend 950. Older NVIDIA Hopper chips are not supported.
03How to Get Started With DeepSeek DeepInfer
To use DeepSeek DeepInfer, you need to set up the right hardware. The episode notes that the smallest efficient setup uses 128 chips: 64 for reading prompts and 64 for writing answers. This is not something you run on a laptop.
- Go to DeepSeek's release page where the code is posted. The engine had over 22,000 stars on GitHub within a day.
- Check your hardware: You need NVIDIA SM100 or Huawei Ascend 950 chips. No other chips are supported yet.
- Download DeepSeek Recipe along with the engine. This tokenizer matches DeepSeek's official API so your model behaves the same as on DeepSeek's service.
- Understand what is missing: The KV cache system and load balancing are not included. DeepSeek wrote a blog post explaining how they work, but you need to build your own.
- Test with the DeepSeek V4.1 Flash model — it's tuned for agentic tasks and performed well on the DeepSWE coding test (77% solved).
04How to Use DeepSeek DeepInfer in Practice
The episode walks through how you can use DeepSeek DeepInfer with agents. DeepSeek V4.1 Flash is a cheap, fast model tuned for agentic jobs. You can plug it into tools like Claude Code, Hermes, or OpenClaw.
The engine lets you swap models behind your agents. For bulk jobs, use the Flash model. For harder tasks, keep a bigger model.
DeepSeek also opened a search API that returns up to 50 ranked web results per search and a fetch API that turns a web page into clean text. These make agents smarter without extra cost. The episode's host uses an agentic operating system called the AirPods (or Airboardroom) to manage all agents in one dashboard.
But you can also use DeepInfer directly if you have the hardware and code.
Structured outputs are especially useful for agents. For example, if your agent needs to produce a JSON object with a name, date, and price, DeepInfer can enforce that format every time. This prevents errors when passing data to other tools.
05Tests, Results, and Comparisons From the Episode
DeepSeek shared test scores for DeepSeek V4.1 Flash, which runs on DeepInfer. On the coding benchmark DeepSWE, Flash solved 77% of tasks. The September version solved 74.2%.
For comparison, Claude Opus 5 scored 74% on the same test. However, these are DeepSeek's own tests. In 7 out of 11 other benchmarks, Anthropic's models still win.
On Humanity's Last Exam, Claude leads by almost 14 points.
DeepSeek says Flash is not the smartest model — it is a small, cheap, fast worker good for agentic tasks. The engine also provides deterministic sampling, so you can run the same test twice and get identical results. That is great for debugging agent pipelines.
The speed charts DeepSeek released ran on their own chips. Nobody outside has verified the numbers on Huawei chips yet. All benchmarks are based on NVIDIA hardware.
06Tips, Limits, and Mistakes to Avoid With DeepSeek DeepInfer
Know your hardware: DeepInfer only works on NVIDIA SM100 and Huawei Ascend 950 chips. If you have older NVIDIA Hopper chips, it won't run. Do not try to run this on a laptop — the smallest setup needs 128 chips.
The KV cache system and load balancing are not included. DeepSeek says those parts are tightly tied to its own systems. You must build your own short-term memory and request routing.
A blog post on their site explains how they work, but it is not code you can run.
Numbers are DeepSeek's own. The 4-7% throughput gain, the 77% DeepSWE score — these have not been independently verified. Test yourself before relying on them for production.
DeepSeek DeepInfer was built on top of VLM, an open-source engine, and borrowed ideas from SG Lang and NVIDIA TensorRT LM. It is not a completely new creation. Open source helped build it.
07Who DeepSeek DeepInfer Is For and Why It Matters
DeepSeek DeepInfer is for AI developers, companies running agents, and anyone who wants to lower inference costs. If you run models at scale, this engine can cut prefill work nearly in half and improve throughput by 4-7%. That adds up fast when processing millions of tokens.
It matters because the edge is no longer owning the best model. It is knowing how to use models well — which jobs to give them, how to hook them into your business. DeepSeek is pushing the whole market toward faster and cheaper AI.
The engine is open source, so other companies can study it and improve it. Future DeepSeek models will also land in the same repository, so you get the engine ahead of time.
For business owners, this means you can run cheap models like DeepSeek V4.1 Flash on your bulk agent tasks and save the expensive models for critical work. That makes AI automation more accessible.
08DeepSeek Recipe: The Second Gift in the Release
Along with DeepSeek DeepInfer, DeepSeek released DeepSeek Recipe. This is a tokenizer — the set of rules that converts your text into tokens and back. If another company uses DeepSeek Recipe, the model behaves exactly like it does on DeepSeek's own API.
Previously, different hosts could produce slightly different results. Now they can match DeepSeek exactly.
Why this matters: If you switch from one provider to another, the model's responses stay consistent. This is especially important for agents that rely on predictable outputs. Also, DeepSeek said that inference code for future models will land in the same place.
You don't have to guess how to run the next model — the engine will be there.
DeepSeek Recipe is built into the DeepInfer release. You can download it together and use it to ensure your system matches DeepSeek's official behavior.
