AI News

Gemini 3.8 Flash Cyber Benchmarks Explained: CyberGym, CWE-Bench, Fairwind Access & What Google Hasn’t Disclosed

Gemini 3.8 Flash and Gemini 3.8 Flash Cyber are distinct deployments. They share what Google calls the same foundational intelligence, but their access paths, safety controls and published evaluation sets differ. The public Flash model is a general-purpose, generally available API model. The Cyber deployment relaxes some cyber-specific safeguards for vetted defenders and is available only through Google DeepMind’s Fairwind Program.

That distinction matters. Google has published the regular model’s API ID, context window, modalities and token prices. It has not published a corresponding Cyber rate card, complete specification, generally available endpoint or public API ID. Google’s “Flash speed and cost” language describes the vendor’s intended positioning; it is not a price customers can audit.

Kingy verdict: Gemini 3.8 Flash Cyber has a credible case for controlled evaluation by mature defensive-security teams. Its strongest public evidence is competitive rather than dominant: 86.2% on a CyberGym vulnerability-reproduction setting and 47.2% on the held-out CWE-Bench audit-and-patch benchmark. The larger constraint is procurement. Most organizations cannot buy or reproduce the deployment today, and several attention-grabbing real-world claims lack the detail needed for independent verification.

Gemini 3.8 Flash versus Gemini 3.8 Flash Cyber

Field Gemini 3.8 Flash Gemini 3.8 Flash Cyber
Deployment General-purpose public model Cyber-specialized, gated deployment with more permissive cyber mitigations
Access Gemini API, Google AI Studio and other documented Google surfaces Approved Fairwind organizations and users only
Public model/API ID gemini-3.8-flash Not disclosed
Public context window 1,048,576 input tokens; 65,536 maximum output Not disclosed for the Cyber deployment
Documented modalities Text, image, video, audio and PDF input; text output Not disclosed for the Cyber deployment
Public price Introductory and post-introductory token rates are published No public Cyber rate card; “Flash-level cost” is a vendor characterization
Evaluation emphasis Broad coding, reasoning, agent and multimodal suite CyberGym, CWE-Bench, Gray Swan indirect prompt injection and a private 20-language vulnerability set
Public model card Available No separate public Cyber model card found as of September 2, 2026

Sources: Google’s Gemini 3.8 launch, Gemini 3.8 Flash API documentation, Gemini 3.8 Flash model card and the Fairwind Program. For the public model’s broader performance, price and migration case, read Kingy’s Gemini 3.8 Flash review.

The blank cells are intentional. Shared foundations do not establish identical context limits, modalities, service terms, retention behavior or endpoint semantics. Copying the regular Flash specifications into the Cyber column would convert an inference into a product fact.

The benchmark results at a glance

Evidence set Gemini 3.8 Flash Cyber result Best comparison point What it supports
CyberGym, Final-submission setting 86.2% Pass@1 GPT-5.5-Cyber 85.6%; Mythos 5 83.8%; GPT-5.6 Sol 83.6%; Gemini 3.5 Flash Cyber 77.5% Strong known-vulnerability reproduction under different owners’ harnesses
CWE-Bench v0 47.2% Pass@1; $3.64 mean cost per rollout Fable 5: 47.8%, $10.27; GPT-5.6 Sol: 44.2%, $2.29 Competitive held-out repository auditing and patching, but not the top score or lowest cost
Google private 20-language set 71.0% recall Gemini 3.7 Flash 58.9%; Gemini 3.5 Flash Cyber 46.6% Material gain on Google’s non-public historical-vulnerability set
Gray Swan indirect prompt injection 6.0% ASR@15 Gemini 3.8 Flash 5.5%; Claude Opus 5 4.8% Low attack success in this transfer-only, no-computer-use configuration; lower is better

All four rows come from Google’s launch package, but they do not have the same provenance. CWE-Bench is an external held-out benchmark run by its operators. Gray Swan is a partner-run evaluation with a disclosed configuration. Google ran CyberGym for its own model in an internal Antigravity harness while comparison scores came from other owners. The 20-language set is entirely private. Treating the table as one homogeneous leaderboard would overstate the evidence.

CyberGym Pass@1: impressive, but not “86.2% of vulnerabilities found”

Google reports 86.2% Pass@1 for Gemini 3.8 Flash Cyber on CyberGym. That is 8.7 percentage points above Gemini 3.5 Flash Cyber, 2.6 points above GPT-5.6 Sol, 2.4 above Mythos 5 and 0.6 above GPT-5.5-Cyber.

Model CyberGym Pass@1 Gap versus 3.8 Flash Cyber
Gemini 3.8 Flash Cyber 86.2%
GPT-5.5-Cyber 85.6% -0.6 pp
Mythos 5 83.8% -2.4 pp
GPT-5.6 Sol 83.6% -2.6 pp
Gemini 3.5 Flash Cyber 77.5% -8.7 pp

The label needs unpacking. CyberGym contains 1,507 tasks across 188 C and C++ projects. Its Level 1 task gives an agent a vulnerability description and an unpatched repository, then checks whether the submitted proof of concept reproduces the issue before the patch and no longer does so after the patch. In other words, the agent is not starting with a clean repository and independently discovering an unknown flaw. It is reproducing a disclosed vulnerability under benchmark conditions.

Pass@1 means success on the first submitted attempt under the stated evaluation protocol. Google’s methodology note says its results use CyberGym’s Final-submission setting and do not use majority voting or parallel test-time compute. That is useful, but it does not make the comparison model-only. Google self-computed its result using an internal, non-cyber-specialized Antigravity harness. The rival results came from their owners, model cards or public leaderboards, and those owners told Google they used proprietary harnesses.

The reasonable conclusion is narrow: Gemini 3.8 Flash Cyber is highly competitive at one-shot vulnerability reproduction in C/C++ repositories. The 0.6-point lead over GPT-5.5-Cyber is too small to support a sweeping superiority claim when harnesses, sampling variance and owner-run pipelines differ.

CWE-Bench: the cleaner patching signal and what each rollout cost

CWE-Bench v0 is the strongest purchasing evidence in the launch package. Collinear AI and Artificial Analysis use 100 held-out audit-and-patch tasks spanning 54 Common Weakness Enumerations and six language categories: C/C++, Go, JavaScript/TypeScript, Java, Python and Rust. The agent receives a repository and an instruction to audit and fix what it finds. It is not told the vulnerability or how many defects are present.

A submission passes only when the vulnerability’s exploit test no longer succeeds and the project’s existing tests still pass. That all-or-nothing verifier is closer to a maintainer’s job than a vulnerability-description benchmark.

Model and harness CWE-Bench Pass@1 Mean billed API cost per rollout
Fable 5 with Claude Code 47.8% $10.27
Gemini 3.8 Flash Cyber with Antigravity 47.2% $3.64
GPT-5.6 Sol with Codex 44.2% $2.29
Gemini 3.7 Flash with Antigravity 44.0% $1.43
Claude Opus 4.8 with Claude Code 42.0% $2.43

Gemini 3.8 Flash Cyber finished 0.6 points behind Fable 5 while costing about 64.6% less per rollout. That is the favorable comparison Google emphasizes. It is not the whole cost story. The Cyber system cost about 59% more than GPT-5.6 Sol for a three-point score advantage, and about 154.5% more than general Gemini 3.7 Flash for a 3.2-point gain. Those figures are Kingy calculations from the leaderboard’s published rounded costs.

“Cost per rollout” is not a Cyber token tariff. CWE-Bench defines it as the mean billed API spend for one agent rollout on one task. The figure therefore reflects model pricing, token use and harness behavior in that run. It excludes engineering labor, sandbox infrastructure, review time and the cost of false positives or unsafe patches. The $3.64 result is real benchmark evidence; it is not a rate card a customer can use to forecast production spend.

The same system-level caveat applies to performance. CWE-Bench compares Gemini-plus-Antigravity, GPT-plus-Codex and Anthropic-plus-Claude Code. Different proprietary harnesses can change context construction, tool use, iteration, test execution and stopping behavior. Buyers are evaluating a deployed agent stack, not an isolated checkpoint.

Google’s private 20-language evaluation: a big gain with a low replication ceiling

Google reports 71.0% recall for Gemini 3.8 Flash Cyber on a private set of more than 1,200 recent, confirmed historical vulnerabilities in popular open-source projects. Gemini 3.7 Flash scored 58.9% and Gemini 3.5 Flash Cyber 46.6% in the same chart. Google says the dataset spans 20 programming languages and that the models ran in an internal, non-cyber-specialized Antigravity harness.

That is a 12.1-point gain over the public 3.7 Flash model and a 24.4-point gain over the prior Cyber deployment. It is also the least reproducible headline benchmark.

Google has not published the repository and vulnerability list, the 20-language breakdown, language weighting, severity distribution, prompts, agent budget, number of runs, false-positive treatment, per-language recall, precision, confidence intervals or cost. “Recall” indicates how many known target vulnerabilities the system recovered, but without the denominator composition and precision, it does not tell a buyer how much review noise the system generated.

The result is best read as an encouraging internal regression signal. It should not be treated as an external proof that the model will find 71% of a company’s unknown vulnerabilities.

Gray Swan indirect prompt injection: lower is better

Gray Swan’s indirect prompt-injection evaluation reports attack success rate, so the correct direction is down. At 15 attempts, Gemini 3.8 Flash Cyber recorded 6.0% ASR@15. General Gemini 3.8 Flash did slightly better at 5.5%, while Claude Opus 5 led the shown field at 4.8%. Cyber therefore ranks near the front of this evaluation, but it does not beat the general 3.8 deployment.

Model Indirect prompt-injection ASR@15 Direction
Claude Opus 5 4.8% Lower is better
Gemini 3.8 Flash 5.5% Lower is better
Gemini 3.8 Flash Cyber 6.0% Lower is better
Claude Fable 5 6.5% Lower is better
Claude Sonnet 5 6.7% Lower is better
Gemini 3.7 Flash 9.2% Lower is better

This was not a computer-use test. Google says it supplied model checkpoints to Gray Swan, which independently configured and ran a combined attack set using transfer attacks only and no computer use. That design tests whether hostile instructions embedded in untrusted content can redirect an agent within the specified environment. It does not cover every browser, terminal, credential or long-running tool-use failure mode.

The result is welcome because a cyber model with relaxed safeguards still needs strong resistance to instruction hijacking. It also argues against a simplistic “more permissive means less secure” story: this deployment remained competitive on the disclosed test. For a broader explanation of why prompt-injection leaderboards cannot be casually mixed, see Kingy’s coding-agent prompt-injection benchmark crosswalk.

Chrome, Wiz and the under-two-hour discovery story

Google’s launch adds three operational claims. They are relevant, but they are case evidence rather than public benchmarks.

Chrome: 2.6 times more correct patches

Google says the Chrome Security team used Gemini 3.8 Flash Cyber to generate 2.6 times more correct patches than the best much-larger commercial models. Google has not named the comparison models, task count, repository slice, harness, definition of “correct,” pass rate, human-review burden, cost or uncertainty. The named internal team makes the account credible enough to investigate. The missing denominator makes it unsuitable for procurement math.

Wiz: higher recall at lower cost

According to Google, Wiz measured 7.5% to 9.7% higher recall on an internal penetration-testing benchmark at 2.3 to 5.2 times lower cost than other leading frontier models. The public wording does not establish whether 7.5–9.7% means percentage points or a relative percentage. It also omits the baseline models, task mix, sample size, harness, pricing basis, precision and false-positive rate. Until Wiz or Google publishes the study, quote the range as reported rather than converting it.

A critical vulnerability found in under two hours

Google says its Cloud Vulnerability Research team found a critical foundational vulnerability with the model in under two hours after months of prior investigation. The affected product, issue identifier, timeline, human contribution, validation method, patch status and disclosure status are not public. Responsible disclosure may explain some of that silence, but it also prevents independent verification today.

The defensive takeaway is not that every hard vulnerability is now a two-hour task. It is that a specialist model may compress the final phase of an expert-led investigation. Organizations should ask Google for a redacted case study, including time accounting and the roles of the model, harness and researchers.

Kingy applies the same standard to other vendor security claims. Our GLM-5.3 cybersecurity claim audit explains why a discovered bug, a confirmed exploit path and a shipped fix are different evidence events.

Finding, validating, patching and exploiting are different jobs

Security-model announcements often collapse four distinct stages into one word: “cyber.” Buyers should separate them.

Stage What the system must establish Relevant evidence here What the result does not prove
Finding Identify a plausible weakness in code or system behavior Google’s private historical-vulnerability set; the audit phase of CWE-Bench That the issue is reachable, exploitable or important
Validating Reproduce the failure and distinguish a real issue from noise CyberGym Level 1; CWE-Bench’s hidden verifier Safe operation in a live environment or broad discovery ability
Patching Produce a code change that removes the issue without breaking existing behavior CWE-Bench; Google’s Chrome claim Production readiness without maintainer review and integration testing
Exploiting Convert a weakness into unauthorized impact Not directly established by the reported patching results Permission, safety or success in a real target environment

CyberGym’s proof-of-concept task is controlled vulnerability reproduction. CWE-Bench combines open-ended auditing with repair and a deterministic test. Neither result authorizes deployment against systems the operator does not own or have permission to test. This article intentionally omits exploit code and operational intrusion guidance.

What is external, internal or partner-reported?

Claim or result Evidence class Operator / source Publicly reproducible?
CWE-Bench 47.2% and $3.64 Public external benchmark Collinear AI / Artificial Analysis The leaderboard and method are public; tasks are held out and Cyber access is gated, so most outsiders cannot rerun it
CyberGym 86.2% comparison Public benchmark, owner-run model result Google ran its model; other owners supplied comparison results Benchmark is public, but harnesses differ and the Cyber model is not generally accessible
Gray Swan 6.0% ASR@15 Partner-run external evaluation Gray Swan, using Google-provided checkpoints Configuration is summarized; checkpoints and combined set are not generally available
Private 20-language 71.0% recall Google-internal evaluation Google No
Chrome 2.6× patch claim Google-internal case evidence Chrome Security, reported by Google No
Wiz recall/cost ranges Partner-reported case evidence, relayed by Google Wiz No public study found as of the cutoff
Under-two-hour vulnerability story Google-internal case evidence Google Cloud Vulnerability Research No; issue details are withheld

“External” does not always mean fully reproducible, and “public benchmark” does not mean every score was generated in one neutral harness. Those distinctions should appear in any internal risk memo or board presentation that cites the launch.

Benchmark limitations buyers should carry into a pilot

The disclosed scores are useful, but six limits prevent a simple ranking:

  1. Harnesses are part of the product. Antigravity, Codex, Claude Code and owner-built CyberGym pipelines can differ in prompt construction, tool policies, context management, retry behavior and stopping rules.
  2. Most outsiders cannot access the Cyber deployment. A public task set does not create a reproducible result when the tested model and its serving configuration are gated.
  3. Pass@1 is protocol-dependent. It describes one accepted submission under a defined run, not the probability that any production job will succeed on its first conversational turn.
  4. Small gaps need uncertainty. Google publishes point estimates, not confidence intervals, for the CyberGym chart. A 0.6-point gap should not outweigh workflow fit or local testing.
  5. Recall is not enough. The private 20-language chart does not report precision, review burden or per-language behavior.
  6. Historical tasks are not novel production incidents. Known vulnerabilities and hidden tests reduce contamination and scoring problems, but they cannot represent every architecture, dependency, build system or organizational control.

The practical answer is a permissioned pilot on the organization’s own resolved historical issues. Freeze the harness, model version, tool permissions, time budget and cost accounting. Measure verified findings, safe patches, regression rate, false positives, maintainer acceptance and analyst time, not only benchmark Pass@1.

Fairwind access: who qualifies and what Google requires

Gemini 3.8 Flash Cyber is distributed through the Fairwind Program, not a self-service model picker. Google says the program prioritizes:

  • governments and national cyber authorities;
  • critical-infrastructure organizations in sectors such as health, telecommunications, energy and finance;
  • core technology platforms and software maintainers; and
  • academic labs conducting defensive cybersecurity benchmarking.

Applicants undergo organizational vetting and due diligence covering ethical conduct and security history. Approval is not transferable to every contractor, affiliate or community member.

Fairwind permits defensive dual-use work such as authorized threat simulation, reverse engineering and malware analysis when performed for defensive or academic purposes. It does not permit malicious activity or creation of malware for harm. The program’s policy is about authorization and purpose, not merely whether a task looks technical.

Approved organizations must provide user-level authentication, phishing-resistant multi-factor authentication and applicable access controls. Access should be limited to internal cybersecurity, incident-response and penetration-testing personnel, with use and entitlements tracked. Credentials or access cannot be shared, redistributed or sold.

Zero data retention is available, with conditions

Google says zero data retention is supported when the model is accessed directly as a managed model on Gemini Enterprise Agent Platform. The platform’s ZDR documentation still requires configuration discipline: abuse-monitoring prompt logging may need an approved exception; request-response logging must be disabled; stored interactions must be turned off; and some grounding or advanced features can prevent ZDR eligibility. An in-memory cache lasting up to 24 hours can still qualify because it is not retained at rest, and customers can ask for caching to be disabled.

Do not convert “ZDR available” into “every Fairwind workflow retains nothing.” Confirm the exact service path, logging exception, region, feature set and contract with Google before placing proprietary code or incident data in the system.

CodeMender is the agent harness, not another name for the model

CodeMender is Google’s code-security agent. It wraps an underlying Gemini model with repository analysis, tools and a workflow that finds candidate vulnerabilities, verifies them and proposes patches for human review. The original CodeMender research announcement predates Gemini 3.8 Flash Cyber and described a system using Gemini Deep Think. The current preview can use multiple Gemini models; approved Fairwind partners can select the Cyber deployment.

That relationship matters when interpreting results. A model supplies reasoning and code generation. CodeMender supplies orchestration, repository context, analysis tools, verification and the patch-review workflow. Improvements attributed to a deployed agent may come from either layer.

CodeMender is in Public Preview for a limited set of customers, and Google’s pre-GA terms say it is intended for testing and evaluation rather than production use. Its documentation lists publicly available model options including Gemini 3.7 Flash as the default, alongside other Gemini releases. Developers who are not Fairwind-eligible can therefore:

  • apply for the CodeMender preview and use a public supported model if admitted;
  • use general Gemini 3.8 Flash through its documented public interfaces for code review and remediation assistance, keeping security-sensitive actions behind human approval; or
  • continue with conventional static analysis, dependency scanning, fuzzing and maintainer-reviewed patch workflows.

CodeMender’s retention behavior is also not identical to direct managed-model ZDR. Google says CodeMender session state can persist for up to seven days during a scan, with source cleared after a terminal state and session records subject to a seven-day deletion window. Teams with strict no-retention requirements should obtain a service-specific answer instead of inheriting the managed-model claim.

Who should apply to Fairwind and who cannot realistically use it

Apply if your organization fits Google’s stated priority groups, already operates a mature defensive-security program, can enforce phishing-resistant MFA and individual entitlements, and has a concrete authorized workload. Strong candidates include national cyber defenders, critical-infrastructure response teams, major platform security groups, high-impact open-source maintainers and academic labs with controlled evaluation plans.

The best application is specific: a bounded repository set, named operators, documented authorization, a safe execution environment, a patch-review process, retention requirements and metrics for verified risk reduction. “We want the strongest cyber model” is not an evaluation plan.

Do not plan around the model if you are an individual hobbyist, a small team without organizational security controls, an unvetted consultancy seeking shareable credentials, a buyer requiring instant self-service access, or anyone whose intended activity lacks written authorization. Ordinary API customers also cannot assume their Gemini key will expose the Cyber deployment. There is no generally available endpoint to build against today.

Evidence-quality scorecard

Evidence item Quality Why
CWE-Bench patching and cost A- / high External operator, held-out tasks, deterministic verifier and exact cost; private set and different agent harnesses limit full replication
CyberGym comparison B / moderate Public benchmark and clear protocol, but Google self-ran its model and rivals used different proprietary harnesses; task is reproduction, not pure discovery
Gray Swan indirect prompt injection B+ / moderate-high Independent partner execution and disclosed transfer-only/no-computer-use setup; gated checkpoints and combined set prevent broad reproduction
Private 20-language recall C / low-moderate Large stated sample and useful within-family comparison, but no public task list, precision, language breakdown, budget or uncertainty
Chrome patching claim C- / low Named internal team and concrete multiplier, but missing baseline, denominator, method and raw results
Wiz recall and cost claim C- / low Named partner and bounded ranges, but no public report, baseline models, task count, precision or cost method
Under-two-hour discovery C- / low Concrete elapsed-time story from a relevant team, but the issue, process and outcome are not independently inspectable
Fairwind access and controls A / high Direct Google program and platform documentation; contract-specific terms still require customer confirmation

The scorecard rates the evidence, not the model’s intrinsic capability. A low grade means the public record cannot carry a strong claim—not that the underlying event did not happen.

Kingy methodology

Kingy reviewed the sources available on September 2, 2026: Google’s launch announcement, Gemini 3.8 Flash API documentation and model card, the Fairwind program and ZDR pages, Google’s four-page Cyber evaluation methodology, the public CyberGym and CWE-Bench descriptions and leaderboards, Gray Swan’s indirect prompt-injection documentation, and Google’s CodeMender research and product documentation.

We transcribed chart values from the published launch materials and checked arithmetic independently. Percentage-point differences use the displayed scores. Cost comparisons use rounded public dollar values and are therefore approximate. We kept Google-run, externally run and partner-reported results in separate evidence classes. No Kingy staff member had Fairwind access, ran Gemini 3.8 Flash Cyber or attempted to reproduce the security tasks. No undisclosed product specification was inferred from the public Gemini 3.8 Flash model.

This is a defensive-security analysis. It contains no exploit code, target-selection guidance, persistence instructions or operational intrusion procedure.

FAQ

Is Gemini 3.8 Flash Cyber the same model as Gemini 3.8 Flash?

No. Google says they share foundational intelligence, but they are separate deployments with different safety controls, access paths and evaluation sets. Google has not documented that every public Flash specification carries over to Cyber.

Can I call Gemini 3.8 Flash Cyber through the public Gemini API?

Not as a generally available model. Access is limited to approved Fairwind participants, and Google has not published a public Cyber API ID or self-service endpoint.

How much does Gemini 3.8 Flash Cyber cost?

Google has not published a Cyber rate card. The $3.64 CWE-Bench figure is average billed API cost for one benchmark rollout, not a token price. “Flash-level cost” is Google’s characterization, not a customer-verifiable tariff.

Did Gemini 3.8 Flash Cyber win CyberGym and CWE-Bench?

It led the five-model CyberGym comparison Google showed at 86.2%, although the 0.6-point lead over GPT-5.5-Cyber came from different owners’ proprietary harnesses. It did not lead CWE-Bench: Fable 5 scored 47.8% versus the Cyber deployment’s 47.2%, at a much higher mean rollout cost.

Is a lower Gray Swan score better?

Yes. The metric is attack success rate. Gemini 3.8 Flash Cyber’s 6.0% ASR@15 was strong, but general Gemini 3.8 Flash scored a slightly lower 5.5% and Claude Opus 5 scored 4.8% in the disclosed chart.

Can ordinary developers use CodeMender without Fairwind?

Potentially, if admitted to CodeMender’s limited Public Preview, using one of its public supported Gemini models. That does not grant access to Gemini 3.8 Flash Cyber. General Gemini 3.8 Flash remains the documented public option.

Can an outsider reproduce the Cyber benchmark results?

Not fully. CyberGym is public and CWE-Bench publishes its methodology and leaderboard, but the Cyber deployment is gated. CyberGym comparison runs used different proprietary harnesses, CWE-Bench tasks are held out, Gray Swan’s checkpoints are not public, and Google’s 20-language set is private.

Does Fairwind guarantee zero data retention?

ZDR is available for direct managed-model access on Gemini Enterprise Agent Platform when the required logging and feature conditions are met. CodeMender and optional platform features have separate retention behavior. Confirm the exact configuration and contract before sending sensitive material.

Update log

  • September 2, 2026: Initial publication. Added CyberGym, CWE-Bench, Google’s private 20-language evaluation, Gray Swan ASR direction, Fairwind controls, CodeMender access paths and a disclosure-gap audit. Research cutoff: September 2, 2026.