Send us Fan Mail

Almost every input token was cached, yet cached-input charges still made up most of the token bill. In Fireworks' DeepSWE run of DeepSeek V4.1-Flash, 99.6% of input tokens were cached and those charges accounted for 59.9% of the $0.4295 per-task token bill. One run doesn't establish a universal cost ratio. It does give buyers a reason to inspect the cached share of their own spending before negotiating an AI contract.

Augur's analysis separates cache hit rate from spending share, explains why DeepSeek's reported 155 trillion served tokens per day includes cache reuse, and examines what the DeepInfer release leaves for operators to build. Its 96% average input-cache hit rate is a company self-report, not an independent audit. DeepInfer supports offline inference on specialized hardware; it is not a turnkey copy of the complete service.

The strongest counterargument matters: reliable cached service can be worth paying for. A high cached share doesn't prove overcharging, and open code doesn't promise savings from running it yourself. Compare the cost of successful work at the quality and response time you require.

Signals on the clock follows the relevant DeepInfer watch. vLLM's published comparison measures its own engine against DeepSeek's published decode results, not an independently operated DeepInfer service. No qualifying outside DeepInfer run was found in the bounded research, which is not proof that none exists.

Augur call from the written dispatch, relevant to this episode:
Augur calls are dated, testable predictions tracked on the site. The original wording, criteria, deadline, and odds follow; this episode assigns no result.

- Call: A named organization other than DeepSeek publishes measured throughput or cache-hit results from running DeepInfer on hardware it controls by 2027-01-31.
- Criteria: A public blog post, paper, or repository from an identifiable company, lab, or inference provider reporting its own DeepInfer measurements on or before the due date counts as a hit; absence of any such publication counts as a miss.
- Due: 2027-01-31
- Odds: 70%

For a business or technology leader: what share of your agent's token bill goes to cached input, and would that change which rate you negotiate?

Support the show

Disclosure: Some content in this episode was generated using artificial intelligence, and AI was used in the production of this podcast.

Follow Augur Dispatch for the daily brief and its accountable calls.