DeepSeek V4.1 Flash vs GPT-6 and Claude: Costs, Benchmarks and Failures

GPT-6 Astra passed all eight trials and had the shortest median API waiting time. DeepSeek V4.1 Flash passed seven at the lowest cost. Claude Fable 5.1 passed six: it read our chart correctly twice, but added prose that broke the requested strict JSON format.

We tested DeepSeek V4.1 Flash, GPT-6 Astra and Claude Fable 5.1 on four small tasks, with two runs of each task per model. The results show useful cost and integration differences. They do not reproduce DeepSeek’s launch benchmarks or settle which frontier model is best.

Tested September 10, 2026. These 24 trials are a subset of one 32-trial campaign that also tested Qwen3.8-27B. The DeepSeek runs are shared with our Qwen comparison.

Our measured results

All spending includes unsuccessful attempts. Cost per passing trial divides the total model bill by its number of passing trials. The median time sums HTTP response waiting time within each task, including network and provider processing; it excludes local grading and billing lookups. It is not a throughput or time-to-first-token benchmark.

All three routes passed both coding tasks on both attempts. The differences came from the small integration tasks: DeepSeek missed one ledger run, and Claude missed the strict formatting requirement on both chart runs. Eight attempts per model are too few to turn those counts into a general reliability ranking.

The failures: a cursor value and a JSON prefix

DeepSeek’s second ledger trial returned {"cursor":"null"}, supplying a string where the first-page request needed JSON null. It repeated that cursor after an error and finally tried an empty string. It never read a page or produced a final answer within four responses. Its first attempt completed the same workflow successfully.

That trace supports a narrow finding about the tested gateway route. We did not isolate every model and adapter component or test whether more turns would recover the job. The task stopped at the limit fixed before the run.

Claude returned the correct chart values on both attempts. Each answer also included explanatory prose before its JSON object, despite the prompt asking for only JSON. We counted both attempts as failures under the predetermined output contract. The first response was:

Eligible: Cobalt (7 leads, $84 → $12/lead), Amber (10 leads, $90 → $9/lead), Jade (8 leads, $64 → $8/lead).

{"eligible_names": ["Amber", "Cobalt", "Jade"], "winner": "Jade", "total_spend_usd": 238, "total_qualified_leads": 25, "blended_cost_per_lead_usd": 9.52}

That distinction matters. A human could read the answer and understand it. Our application contract required json.loads(answer) to work on the entire response. A prefatory sentence breaks that contract even when the following object is correct.

We tested instruction-following on ordinary chat completions. We did not enable provider-enforced JSON schemas or add a repair layer. In an application, those are sensible options to investigate where supported. This result is not evidence that Claude cannot interpret the chart or that its structured-output feature failed.

What the coding and tool checks establish

The first coding task repaired interval merging: preserve inputs, combine overlapping or touching windows, keep nested ranges intact, discard empty intervals and reject invalid ones. Nine checks assessed each answer.

The second repaired proportional discount allocation. It required exact integer arithmetic, deterministic tie-breaking and correct behavior above JavaScript’s familiar 2⁵³ precision boundary. Thirteen checks assessed each answer. Passing these functions is useful; it is a much smaller demand than resolving an unfamiliar repository issue.

The tool task required recovering from a retryable mock error, reading two ledger pages, deduplicating an invoice and excluding void or refunded records. Every operation stayed inside a synthetic fixture. It tested a short recovery sequence, not a long autonomous session.

DeepSeek’s launch benchmarks tell a more mixed story

The official V4.1 table compares with GPT-5.6 Sol and Claude Opus 5.0, among other models. Those are different opponents from the GPT-6 Astra and Claude Fable 5.1 endpoints in our fresh test. Do not silently substitute the newer names into the launch chart.

Source: DeepSeek’s launch model card at revision df42c109. These are vendor-reported scores, not Kingy reproductions. The HLE row uses the main score, not DeepSeek’s separately reported text-only subset. DeepSeek used maximum reasoning effort for its instruct evaluations; its agentic results also depend on the stated harness.

The table supports a specific claim: V4.1 is competitive on some agentic evaluations at its advertised token prices. It also shows substantial deficits on other evaluations. A small numerical lead on one benchmark does not establish a statistically meaningful general advantage.

Different benchmark versions answer different questions. Leading Terminal-Bench 2.1 while trailing Terminal-Bench 4.0 is a reason to inspect task difficulty and setup. Averaging those percentages with a knowledge exam would produce a convenient number without a clear meaning.

Token prices versus the cost of getting a usable answer

Sources: DeepSeek pricing, OpenAI’s GPT-6 Astra documentation, Anthropic’s model overview and the gateway catalog. Cached input, regional routing, long context and service tiers can change the bill. Our requests used short contexts and pinned standard provider routes.

In our sample, GPT’s cost per passing trial was approximately 7.7 times DeepSeek’s, and Claude’s was 20.0 times DeepSeek’s. Those ratios include each model’s failures and its actual token usage. They apply to these short tasks and settings; they are not savings forecasts for a production agent.

Reasoning effort deserves particular care. All models received the gateway’s low effort setting and a 3,072-token output cap. That does not provide equal internal compute. Reported reasoning-token fields also differ by provider; a zero in the returned usage field does not prove that a model performed no internal reasoning. The billing records are the stronger evidence for this cost comparison.

Open weights change the deployment decision

DeepSeek publishes downloadable V4.1 Flash weights under MIT. GPT-6 Astra and Claude Fable 5.1 are hosted proprietary offerings in this comparison; their model weights are not supplied for a comparable self-hosted installation. Open weights, an API and a fully reproducible training process are different things. The presence of a weight download alone does not establish complete training reproducibility.

V4.1’s own checkpoint is large: our audit found about 510.3GB across 48 official weight files. Its attractive API price does not imply that you can run the official release on a laptop. See the V4.1 local hardware guide for the RAM, VRAM and runtime evidence.

For a smaller model you can evaluate for desktop deployment, our DeepSeek versus Qwen3.8-27B comparison uses the same task campaign and explains the difference between hosted results and local hardware claims.

Which route would we try first?

For short, inexpensive jobs with a deterministic acceptance check, DeepSeek is an attractive candidate. For this exact set, GPT supplied the most consistent combination of passing output and low waiting time, at a higher token cost. Claude’s two format misses are a reason to validate its output contract for this workflow, not a verdict on its broader coding or research capabilities.

For difficult repository work, long research sessions or ambiguous decisions, these four fixtures are insufficient. Use the published benchmarks to choose candidates, then evaluate a few representative tasks with an acceptance test. Include retries, tool rounds and human review time in the economics.

A cheap model that fails your task can be expensive to operate. A premium model that provides no additional value on a simple extraction can also be expensive. The useful unit is a correctly completed job under your application’s rules.

Methods and reproducibility

We fixed the four prompts, fixture image and checks before the paid run. Each model received two fresh attempts per task. Provider order was reversed for repeat two. Sampling remained at provider defaults, so this is not an experiment with an equal temperature or equal compute budget. Providers were pinned to DeepSeek, OpenAI and Anthropic through Vercel AI Gateway; there was no cross-provider fallback.

Each tool trial allowed at most four model responses, including the final answer. There were no paid retries to improve a failed model answer, no answer repair and no LLM judge. All response statuses, finish reasons, charges and outputs are retained.

The downloadable evidence bundle contains the complete 32-trial campaign, including Qwen, with exact requests, responses, grading and billing records. The complete four-model campaign used 56 model calls and cost $0.313674 in confirmed gateway charges, well below our $10 limit. No GPU rental or paid judge was required. Inspecting the files is free; rerunning the API script incurs usage charges.

Right now

Trending on Kingy

Keep reading with the stories getting the most attention now.

1. 01 Blog · Sep 8

   OpenAI Releases a Navier–Stokes Proof Claim: The Evidence and the Dispute

   Read story →
2. 02 Blog · Sep 5

   Blender with OpenAI Astra: The Complete Guide to Creating 3D Models, Animations, Product Visuals, and Virtual Worlds

   Read story →
3. 03 Blog · Sep 10

   Can You Run DeepSeek V4.1 Flash Locally? RAM, VRAM and Mac Requirements

   Read story →