Gemini 3.8 Flash vs GPT-5.6: Our 192-Run Work-per-Dollar Test

Google’s new Gemini 3.8 Flash did not win every column in our test. It won the decision that matters for most buyers.

GPT-5.6 Sol posted the highest mean quality score, 98.6 out of 100. Gemini finished at 97.7, only 0.9 points behind, while costing about half as much per acceptable result: 1.81 cents versus 3.42 cents. Both completed all 24 planned attempts. Gemini also passed our 80-point bar on 23 of 24 attempts.

The open-weight field was more complicated. DeepSeek V4 Flash 0731 cost a remarkable 0.15 cents per acceptable result on the text tasks its route supported. That is about twelve times cheaper than Gemini. But the OpenRouter route had no image-capable endpoint, and its source-contained knowledge score was 79.5. Qwen3.8 Flash looked similarly cheap until four knowledge-work runs spent the entire 4,096-token allowance on reasoning and returned no answer. It also timed out twice and produced one malformed tool call.

Our verdict is workload-specific:

• Best default for high-volume agents and mixed document work: Gemini 3.8 Flash.
• Best raw quality: GPT-5.6 Sol.
• Cheapest useful text route: DeepSeek V4 Flash 0731, if its support matrix and weaker knowledge-work result fit the job.
• Best open-weight quality in this run: Kimi K3, though it cost more than GPT-5.6 Sol and ran much slower.
• Most practical single-GPU checkpoint: Qwen3.8-27B loaded in BF16 on one A100 80GB, but our short RunPod test did not beat its hosted price and exposed a reasoning-template problem.

This is a controlled Kingy workload sample, not a universal leaderboard. We ran 192 planned attempts across eight routes on September 2, 2026, then judged knowledge work blind and executed code tests in a network-denied sandbox. The raw results, task definitions, scoring report, and RunPod result are included with the draft.

For a launch-focused review of Google’s specifications and public benchmarks, read our Gemini 3.8 Flash review. Our Gemini 3.8 Flash vs Muse Spark 1.3 comparison covers that narrower head-to-head. This article answers a different question with eight routes, 192 planned attempts, blind grading, and a rented-GPU check.

The 40-second comparison

Every model received the same 12 tasks twice, medium API reasoning, a 4,096-token completion cap, and price-sorted OpenRouter routing with data collection denied where the platform exposed that control. “Applicable success” excludes attempts the route explicitly did not support. “Strict success” counts every planned attempt, including unsupported image calls and operational failures.

*Mean score covers completed, scored attempts. Use the support and failure columns when judging whether a route can serve as a general-purpose default. DeepSeek V4 Flash and Qwen3.8 2.4T-A95B each lacked an image-capable endpoint for four planned attempts.

The chart uses a logarithmic cost axis because Claude Fable’s $0.2446 per acceptable result would otherwise compress the low-cost routes into the first few pixels. Cheap models can still fail expensively in production. The Qwen Flash figure, for example, divides observed spend by 17 successful attempts, but an application would also need to absorb three failed calls and four empty knowledge answers.

Two unrelated models now wear “3.8 Flash”

Search for “3.8 Flash” and you will encounter two different model families.

Gemini 3.8 Flash is Google’s proprietary September 2 release. The model card lists a one-million-token input window, up to 64,000 output tokens, and text, image, audio, and video input. Google distributes it as a managed service rather than downloadable weights.

Qwen3.8-Flash-Next is Alibaba Qwen’s downloadable August 26 checkpoint. Qwen describes a hybrid architecture that interleaves three gated-delta-network layers with one global-attention layer. Its goal is efficient long-context processing with open weights.

OpenRouter lists a weights-backed route called qwen/qwen3.8-flash. That route returned the exact same route name in our responses and used Alibaba as the provider. The public listing does not establish byte-for-byte parity with the official Qwen/Qwen3.8-Flash-Next repository, so we do not claim it. Hosted route IDs, downloadable checkpoint IDs, provider implementations, quantization, and licenses belong in separate columns.

That naming discipline matters here. The Google model and the Qwen model share a version number and a speed-oriented label. They do not share a vendor, architecture, license, service, or result profile.

Why Gemini 3.8 Flash deserved a real workload test

Google positions Gemini 3.8 Flash as the fast, economical member of its current family. The model card also warns that higher reasoning settings may use more tokens, run more slowly, or time out. Those details matter more to an API bill than a benchmark percentage does.

Artificial Analysis’s launch evaluation gave the model an Intelligence Index score of 59 at high reasoning and found improved agentic performance. It also reported roughly 30% more output tokens than Gemini 3.7 Flash and an average cost of $0.58 per task, about 40% higher than the predecessor despite unchanged per-token pricing. That was the useful hypothesis for our test: stronger reasoning can make a “Flash” model more capable without making every completed job cheaper.

We therefore measured the bill returned by the API, not a price-sheet estimate. OpenRouter’s usage.cost accounting includes the provider’s native charge, so reasoning and cache behavior remain attached to each attempt. We also retained prompt, completion, reasoning, and cached-token counts.

Our results support a narrower conclusion than Google’s launch material or a public index can offer. On this medium-reasoning workload, Gemini produced near-top quality at a low completed-work cost. It did not have the lowest token bill, the lowest time to first token, or the highest score. It combined strong quality, full capability support, and consistent completion better than the other low-cost routes.

What we tested

The suite contains 12 synthetic tasks in six categories:

1. Coding: merge intervals with edge cases, and reconcile ledger records. Model outputs were parsed and executed against hidden fixtures inside macOS sandbox-exec with network access denied.
2. Agent tool use: reorder inventory and place a calendar hold through simulated tools. The scorer checked call order, arguments, recovery from rejected actions, and final state.
3. Knowledge work: write a launch decision memo and a vendor-policy recommendation using only supplied source packs. Three blind judges scored factual accuracy, arithmetic, contradiction handling, evidence discipline, and decision quality. We used the median score.
4. Long context: retrieve exact facts and resolve cross-references from generated records long enough to test attention without importing private or copyrighted material.
5. Multimodal: read an original revenue-and-margin chart and an operations dashboard with a footnote that changed the correct answer.
6. Structured reasoning: return exact invoice and policy data in machine-readable form.

Each route received two independent attempts per task. We set reasoning to medium and capped completions at 4,096 tokens. Medium is a vendor control, not a standard unit of compute. One vendor may spend far more internal reasoning tokens than another.

The run used OpenRouter’s price-sorted provider routing, fallbacks enabled, and a per-route price ceiling. We asked for endpoints that denied data collection according to OpenRouter metadata. require_parameters remained off because our canary found no Gemini endpoint advertising every optional parameter. OpenRouter also did not expose a common hosted-precision label across all eight routes, so we recorded the actual route, returned model, provider, usage, and finish reason instead of pretending every open-weight deployment used the same quantization.

Provider routing did vary. Gemini stayed on Google, GPT stayed on OpenAI, Claude Fable used Google, Qwen Flash used Alibaba, and Qwen 2.4T used DeepInfra. DeepSeek calls landed on Baidu and OpenInference. Kimi calls used five providers; Qwen3.8-27B used AkashML and Chutes. This is a test of cost-optimized hosted routes that a buyer could call, not a checkpoint-purity lab experiment.

The suite is intentionally small. Two attempts can reveal repeatable breakage and large cost gaps, but they cannot establish population-level differences of one or two points. The visual tasks cover still images, not audio or video. The tool tasks use a simulator, not a live calendar or warehouse. Model judges can also share biases, which is why exact fixtures carry more weight than prose grades.

Before the full run, a four-call canary tested Gemini and Qwen Flash on one invoice and one chart. All four answers passed for $0.0065. The full generation run then cost $8.7733. Blind judging raised total observed OpenRouter spend to $9.5182, or about $9.5247 including the successful canary. That stayed below the authorized $25 ceiling.

Quality results before price

GPT-5.6 Sol was the cleanest model in the suite. It passed every coding, tool, long-context, vision, and structured attempt, then averaged 91.8 on the four knowledge memos. Its 98.6 suite mean led Gemini by less than one point.

Gemini 3.8 Flash matched GPT on coding, tools, long context, and structured reasoning. It averaged 92.5 on knowledge work, slightly above GPT’s 91.8, and 93.8 on vision because one operations-dashboard attempt missed part of the exact answer. That single miss produced its 23-of-24 pass count.

Claude Fable 5.1 and Kimi K3 formed the next group. Their means were 94.9 and 94.6. Both completed every request, passed deterministic coding, long-context, and structured tasks, and lost points on agent execution, knowledge work, and one vision attempt. Those are good results. The cost and latency columns keep them from being automatic defaults: Fable consumed $5.14 for this suite, while Kimi’s median request took 15.3 seconds.

DeepSeek V4 Flash scored 93.0 across the 20 attempts its hosted route completed. It passed all coding, long-context, and structured tasks, but averaged 85.4 on tools and 79.5 on knowledge work. Its four image calls failed before inference because OpenRouter could not find a compatible endpoint. Calling that a zero-quality image answer would be misleading. Calling the route multimodal would also be misleading.

Qwen3.8 2.4T-A95B averaged 90.6 on 20 supported completions. It scored 90.5 on knowledge work and passed all long-context and structured tasks. One of four code attempts did not contain parseable JSON, and the agent tasks averaged 87.5. Like DeepSeek, its four image attempts were unsupported.

Qwen3.8 Flash and Qwen3.8-27B exposed a completion-budget failure mode. Qwen Flash used exactly 4,096 reasoning tokens on each of four knowledge tasks, hit finish_reason: length, and returned an empty answer every time. Qwen3.8-27B also hit the limit on all four knowledge tasks. Three outputs were empty; the fourth contained only 497 characters and earned a median judge score of 22. Both routes performed well on many exact tasks, so their low suite averages do not mean they cannot reason. They mean this configuration failed to convert its reasoning budget into a usable answer.

Qwen Flash added three operational failures: a coding timeout after 240 seconds, a vision timeout after 240 seconds, and a calendar call rejected because function.arguments was not valid JSON. Those failures are precisely why cost per generated token is a weak production metric.

Cost per completed task

Gemini’s full generation bill was $0.4159 for 24 attempts, or 1.73 cents per planned attempt. Twenty-three passed our 80-point bar, so the observed cost per acceptable result was 1.81 cents.

GPT-5.6 Sol cost $0.8210 and passed all 24. Its 3.42-cent cost per acceptable result bought the highest quality and the lowest median total latency in the table. That is a sensible premium when errors cost more than tokens.

Claude Fable 5.1 shows how context can dominate a bill. Its four long-context calls cost about $4.14 by themselves. The model remained fast and strong, but the final 24.46 cents per acceptable result was 13.5 times Gemini’s figure in this workload.

Kimi K3 cost 5.77 cents per acceptable result. Qwen3.8 2.4T-A95B cost 5.19 cents. Both are downloadable model families, yet both cost more per successful hosted attempt than GPT in this run. Open weights grant deployment control; they do not guarantee the cheapest managed API.

DeepSeek was the price outlier in the other direction. Its entire 20-completion generation bill was 2.42 cents, and its cost per 80-point result was 0.15 cents. For coding, extraction, or long-context retrieval that fits the route, the economic case is hard to ignore. For a mixed inbox containing screenshots, the route’s missing image support changes the answer before quality scoring begins.

Launch-day list prices moved after the run. At the final staging refresh, OpenRouter listed Kimi K3 at $3 per million input tokens and $15 per million output tokens, DeepSeek V4 Flash 0731 at $0.065/$0.18, and Qwen3.8-27B at $0.425/$2.55. Gemini remained $0.75/$3.75, GPT-5.6 Sol $2/$10 at its promotional rate, Claude Fable 5.1 $10/$50, Qwen3.8 Flash $0.15/$0.47, and Qwen 2.4T-A95B $2/$6. The result table uses the native cost billed for each completed call, so those launch-day list-price changes do not alter the reported cost per pass.

The Qwen Flash bill was only 6.97 cents, but four empty memos and three failed requests cut its applicable success rate to 70.8%. A production system that retries those calls on Gemini or GPT needs to charge both attempts to the job. Our table stops at observed first-call spend, so it is favorable to the unreliable route.

Coding and tool use

Every completed coding response from Gemini, GPT, Claude Fable, Kimi, DeepSeek, and Qwen Flash passed its hidden fixtures. Qwen Flash completed three of four because one ledger request timed out. Qwen 2.4T and Qwen3.8-27B each produced one ledger response without a parseable JSON object, leaving both at three passes from four attempts.

The tool-use tasks created more separation. Gemini and GPT completed all four simulated agent jobs perfectly. Claude Fable, Kimi, Qwen 2.4T, and Qwen3.8-27B each averaged 87.5. They generally reached the right final state but lost credit for an argument, call order, or recovery step. DeepSeek averaged 85.4. Qwen Flash’s three completed tool runs were perfect, but its second calendar attempt never entered the scorer because Alibaba rejected malformed JSON arguments.

This is a small but useful engineering signal. A tool model needs valid arguments on every call, not persuasive prose after a failed call. For a high-volume agent, Gemini’s combination of full completion, perfect tool transcripts, and low cost is more valuable than winning a prose preference by a point.

For a broader controlled coding comparison between two open-weight candidates, see Kingy’s Qwen3.8-27B vs DeepSeek V4 Flash test.

Knowledge work, long context, and images

Knowledge work was the hardest category to score and the most revealing failure mode. The source packs contained corrections, arithmetic, constraints, and conflicting dates. Judges rewarded answers that used the corrected number, respected the evidence boundary, and made a concrete decision without inventing facts.

Gemini led that category at 92.5, followed by GPT at 91.8 and Qwen 2.4T at 90.5. Claude Fable scored 88.0, Kimi 86.5, and DeepSeek 79.5. The difference between 92.5 and 91.8 is noise at this sample size. The difference between a low-90s memo and no answer is operationally large.

All models passed every completed long-context attempt. This does not prove their advertised context ceilings or performance at one million tokens. It shows that each tested route could retrieve and cross-reference the exact facts in our generated records. Cost separated them more than correctness. Claude Fable spent about $4.14 across four long-context calls; Gemini spent about $0.286, GPT $0.711, Kimi $0.807, Qwen 2.4T $0.709, Qwen Flash $0.054, DeepSeek $0.019, and Qwen3.8-27B $0.117.

On still images, GPT and Qwen3.8-27B passed all four attempts. Gemini, Claude Fable, and Kimi each scored 93.8 after one partial operations-dashboard answer. Qwen Flash’s three completed vision attempts passed, while one timed out. Qwen 2.4T and DeepSeek could not be tested because the selected hosted routes exposed no image endpoint.

We did not test audio or video. Google’s model card says Gemini accepts both, but this article makes no hands-on claim about their quality.

Open weights do not make self-hosting free

“Open source,” “open weights,” and “cheap API” describe different things. A downloadable checkpoint gives you deployment control subject to its license. It does not shrink the files, reserve GPUs, select a serving template, protect the endpoint, or keep utilization high.

Sparse mixture-of-experts models make the distinction sharper. Kimi K3’s model card lists 2.8 trillion total parameters. Qwen3.8 2.4T-A95B names both the 2.4-trillion total and 95-billion active counts. Active parameters influence compute per token; total parameters still influence weight storage and resident memory. A giant sparse model can be efficient to run after it has been distributed across an expensive cluster.

Licenses also differ. Qwen’s 2.4T checkpoint uses a custom license with separate conditions for some large service businesses. Kimi K3 also uses a custom license. Review the exact repository license for the checkpoint you deploy instead of treating “weights available” as a blanket commercial permission.

Kingy’s Qwen3.8-Flash-Next hardware guide covers VRAM and runtime choices in detail. The open-weight model guide tracks the broader field. Buy hosted tokens until privacy, control, customization, or steady utilization makes owning the serving stack worth the work.

What our $0.17 RunPod test learned

RunPod had one secure-cloud NVIDIA A100 80GB PCIe available at an API rate of $1.39 per hour. The web interface displayed $1.41 per hour including storage. We created a pod with Qwen/Qwen3.8-27B in BF16, vllm/vllm-openai:latest, no persistent volume, and a declared 65,536-token maximum context.

The exact checkpoint loaded and the first healthy HTTP response arrived after 385 seconds. The full experiment lasted 438 seconds and cost an estimated $0.1691 at the API rate. The interface showed 66% GPU memory, 34% host memory, and 47% disk use during startup.

The invoice probe generated 223 completion tokens in 8.568 seconds and contained the correct JSON. The memo probe generated 1,200 tokens in 42.972 seconds and hit its cap. Both responses exposed internal reasoning and a literal </think> delimiter before the answer. The invoice was recoverable; the memo spent its budget reasoning and never delivered a clean decision memo.

That configuration problem invalidated a direct quality comparison with the hosted route. Fixing the reasoning parser, chat template, and output channels would change the serving setup, so we stopped instead of patching conditions mid-run. We also skipped the planned concurrency-four test. The pod was terminated and verified absent from the active list.

The economic probe still answers a bounded question. Across the two requests, generation averaged 27.61 output tokens per second. Including cold start, the run cost $118.85 per million generated tokens. If startup were fully amortized and that single-stream rate held, compute alone would cost about $13.98 per million output tokens. The hosted Qwen3.8-27B route listed $2.55 per million output tokens.

Matching the hosted output price at $1.39 per GPU-hour requires roughly 151 aggregate completion tokens per second before storage, idle time, engineering, monitoring, and failed jobs. That is about 5.5 times the measured single-stream throughput. Higher concurrency might close part of the gap, but we did not measure it and will not claim it.

Qwen3.8-27B BF16 fits and serves on one rented A100 80GB. This short run did not establish cost parity, and the default serving template was not ready for clean production responses.

Which model should you choose?

Choose Gemini 3.8 Flash for a high-volume general agent. It completed every task, produced perfect code and tool scores, handled images, and cost 1.81 cents per acceptable result. GPT’s quality edge was small in this suite; Gemini’s cost edge was not.

Choose GPT-5.6 Sol when the last point of reliability is worth paying for. It was the only route to pass all 24 attempts at or above 80, led mean quality, and posted the fastest median total time. Its 3.42-cent cost per acceptable result remained modest compared with a human review or a failed downstream action.

Use DeepSeek V4 Flash for low-cost text pipelines with a narrow contract. Coding, structured extraction, and long-context retrieval were excellent for the price. Gate knowledge memos more carefully and route image jobs elsewhere. Its OpenRouter providers changed during the run, so pin and retest a provider if reproducibility matters.

Consider Kimi K3 when open weights and hosted all-round quality matter more than latency. It was the strongest open-weight family by mean score among routes that completed the full multimodal suite. Its 15.3-second median and 5.77-cent cost per acceptable result trail Gemini and GPT.

Treat Qwen3.8 2.4T-A95B as a strong text model, not a cheap universal API. It scored 90.6 on completed tasks and 90.5 on knowledge work, but the tested route lacked image support and cost more per successful result than GPT.

Do not deploy Qwen3.8 Flash with this exact 4,096-token medium-reasoning configuration for knowledge work. Its exact-task performance and price were attractive, but four empty memos and three failed calls are disqualifying until the output-budget behavior is fixed and retested.

Self-host Qwen3.8-27B for control, not because one lightly used A100 is automatically cheaper. The checkpoint fit. The observed economics and serving-template failure did not justify replacing the hosted route for ordinary low-volume use.

Final verdict

Gemini 3.8 Flash moved the price-quality frontier in this Kingy workload. GPT-5.6 Sol remained the quality leader, but Gemini delivered 99.1% of its mean score at 52.9% of its cost per acceptable result. It also completed all 24 attempts, supported every tested modality, and avoided the severe output-budget failures seen in the cheaper Qwen routes.

DeepSeek V4 Flash is the sharper bargain when the task is text-only and tightly scored. Its supported-task cost was in another league. The missing image endpoint and weaker knowledge score keep it from replacing Gemini as the general default.

Start with Gemini for mixed high-volume work, route the hardest or highest-consequence jobs to GPT, and test DeepSeek against a narrow text contract when pennies matter. That routing policy follows the measured strengths instead of forcing one model to do every job.

FAQ

Is Gemini 3.8 Flash open source or open weight?

No. Gemini 3.8 Flash is a proprietary Google model delivered through managed services. Google publishes a model card and API access, not downloadable weights.

Is Qwen3.8 Flash the same model as Gemini 3.8 Flash?

No. Gemini 3.8 Flash comes from Google. Qwen3.8-Flash-Next comes from Alibaba Qwen and has downloadable weights. The shared “3.8 Flash” wording does not imply a technical relationship.

Is OpenRouter’s Qwen3.8 Flash exactly Qwen3.8-Flash-Next?

OpenRouter’s public listing calls qwen/qwen3.8-flash a weights-backed route and dates it to the Qwen release. It does not prove exact checkpoint parity with Qwen/Qwen3.8-Flash-Next. Our results apply to the hosted route ID and Alibaba provider observed on September 2, 2026.

How much does Gemini 3.8 Flash cost?

At our evidence cutoff, OpenRouter listed $0.75 per million input tokens and $3.75 per million output tokens for google/gemini-3.8-flash. Prices and providers can change. Our 24 generation attempts cost $0.4159 in total.

Which tested model was cheapest per successful task?

DeepSeek V4 Flash 0731 was cheapest on supported tasks at about $0.0015 per result scoring 80 or higher. Its selected route could not accept our image tasks. Among models that completed all 24 planned attempts, Gemini 3.8 Flash was cheapest at $0.0181 per acceptable result.

Can Qwen3.8-Flash-Next run on one GPU?

Hardware depends on precision, context, runtime, and the exact checkpoint. See Kingy’s Qwen3.8-Flash-Next VRAM and vLLM guide for the full deployment analysis.

Can Qwen3.8-27B run on one rented GPU?

Yes. We loaded Qwen/Qwen3.8-27B in BF16 on one A100 80GB and served two requests. The cold start took 385 seconds. The default vLLM response exposed reasoning text, so a production deployment needs a correct reasoning parser and template before quality testing.

Are 12 tasks enough to name the best AI model?

No. Twelve tasks run twice can answer a specific workload question and expose large reliability or cost differences. They cannot rank every capability, language, provider, context length, or deployment. Treat these results as a reproducible buying test and rerun the package with your own highest-value tasks.

Testing disclosure and evidence

• Evidence cutoff: September 2, 2026, America/Vancouver.
• Primary run: kingy-wpd-20260902T184546Z.
• Planned attempts: 192; completed: 181; unsupported image attempts: 8; operational failures: 3.
• OpenRouter spend: approximately $9.5247 including the successful canary and blind judging, below the authorized $25 ceiling.
• RunPod spend: estimated $0.1691, below the authorized $5 ceiling.
• No customer data, live tools, affiliate placement, or sponsored ranking was used. The article was staged as an unpublished WordPress draft only after testing and editorial review.
• Route prices, provider availability, licenses, and model cards should be refreshed on publication day.
• Final staging refresh: September 2, 2026, America/Vancouver. Current route-price changes are disclosed in the cost section and recorded in price-refresh-2026-09-02.md.

Reproducibility files: methodology, model identities, source ledger, tasks, raw generations, blind judgments, isolated code scores, capability-aware report, and RunPod experiment.