DeepSeek V4 Flash 0731 IQ2 vs IQ3: Which Quant Should You Download?

Two complete quants, 354 executed case-seed evaluations, one unchanged six-A40 host—and a result too close to justify pretending we tested the whole download ladder.

The short answer

If you are choosing between the two quants we actually measured, download UD-IQ2_M first. Its 90.927GB payload is 13.281GB smaller than UD-IQ3_XXS, while the larger quant led by only two points on a 426-point decision rubric: 320 versus 318. That 0.47-percentage-point gap is not persuasive evidence of a stable quality tier.

IQ3’s small advantage came from tool-call scoring, where it earned 234/300 to IQ2’s 231/300. IQ2 answered one more exact reasoning item, 67/90 versus 66/90. Both scored 20/30 on strict JSON, and neither produced a scorable Python or TypeScript patch in any of three seeds. The result is a task split, not a clean ranking.

This is a narrow recommendation for one pinned runtime and host, not a declaration that IQ2 is the best DeepSeek V4 Flash quant everywhere. If your decision is primarily about platform fit, use Kingy’s separate guide to DeepSeek V4 Flash 0731 local hardware requirements. If you are still choosing the model, see Kimi K3 versus DeepSeek V4 Flash 0731.

What was tested—and what was not

The frozen plan selected five representative downloads from Unsloth’s 13-label repository: UD-IQ1_S, UD-IQ2_M, UD-IQ3_XXS, UD-Q4_K_XL and UD-Q8_K_XL. Their 20 shards total 594,638,870,752 bytes. The execution protocol kept one verified payload at a time on a 300GB volume.

The 59-case base suite was repeated at seeds 731, 1731 and 2731. Per quant, it contained one Python patch, one TypeScript patch, one strict-JSON task, 20 tool-use cases, 30 exact-answer reasoning problems and six multilingual prompts. That produced 177 executed case-seed rows per quant.

The decision denominator was narrower: 159 rows and 426 available rubric points per quant. The 18 multilingual rows were retained as a literal/canonical-marker diagnostic but excluded from the composite, recommendation, ranking and headline. The scorer can reject valid localized formatting or an unlisted synonym, so it does not establish multilingual factual preservation, multilingual degradation or general translation fluency.

The measured IQ2 versus IQ3 result

The most important result is not the two-point lead. It is the absence of a decisive separation. A 13.281GB payload increase bought three additional tool points and lost one reasoning point under this suite. Both quants tied on JSON and both failed the tiny patch sample. The evidence cannot show that IQ3’s extra 14.6% of download bytes creates a repeatable capability tier.

The coding result deserves particular caution. Three seeds around one Python task and one TypeScript task are not a broad software-engineering benchmark. All 12 quant-by-task-by-seed attempts exhausted the 4,096-token output cap without producing visible unified-diff file headers, so neither quant produced a scorable patch. This establishes a completion-and-format failure in these fixtures; it does not establish whether a valid patch would have passed the hidden tests. For coding agents, fixture-specific completion behavior and hidden tests both matter more than an overall percentage.

Length caps and corruption checks

The harness applied nine deterministic checks to every output: invalid UTF-8, replacement characters, unexpected control characters, degenerate spaces, degenerate blank lines, repeated 20-token n-grams, repeated paragraphs, premature end-of-sequence and runaway generation.

IQ2 had 35 outputs with at least one failed check; IQ3 had 34. In both sets, 32 failures were the harness’s runaway-generation condition, mostly outputs that reached their configured token cap. IQ2 recorded three repeated-20-token-ngram failures and one premature-EOS failure; IQ3 recorded two repeated-ngram failures. These counts do not establish a stable failure order—the difference is one output—but they do show why a deployment should enforce token ceilings and repetition monitoring.

Speed and memory on one six-A40 host

The host was an unchanged Linux 6.8 system with six NVIDIA A40 GPUs. nvidia-smi exposed 46,068MiB per GPU, or 289,834,795,008 bytes in aggregate. The machine had 540,644,073,472 bytes of physical RAM, no swap and 96 logical CPUs; llama.cpp used 52 CPU threads. The runtime was llama.cpp b10219, commit c629da565c80b0b17fac6262acdca4d772e745d8.

IQ2’s speed edge is directionally consistent with its smaller payload, but the suite was designed for task behavior rather than controlled performance isolation. There were no page-cache-controlled cold/warm pairs, no independent repeated load protocol and no alternate hardware. Treat the figures as one-run deployment observations.

What the Q8 smoke established

Before scoring, the 161.870GB UD-Q8_K_XL payload completed a generation-free capacity smoke at a 32,768-token allocation. The recorded 39.8083 seconds runs from process start through the controlled server stop; it is not an exact cold-load or load-to-health latency. The server log recorded model loaded at its logger-relative 33.672702-second mark, while health was polled at roughly five-second intervals.

The largest same-timestamp aggregate GPU-used sample was 176,389,357,568 bytes. No generation request was sent, and the smoke did not record system-RAM telemetry. It establishes that this exact host/runtime combination could load the Q8 payload for the smoke—not stable 32K inference, quality, speed or long-context retrieval.

Why Q8 is only 6.8GB larger than Q4

DeepSeek V4 Flash is not a conventional BF16 checkpoint uniformly compressed to one, three, four or eight bits. The pinned official configuration describes a 43-layer mixture-of-experts model with 256 routed experts, six selected per token, native FP4 expert storage and an FP8 configuration elsewhere. DeepSeek reports 284B total parameters with 13B active per token.

Unsloth says the routed experts account for roughly 96% of the model and remain in native MXFP4. Its Q8 and Q4 recipes therefore differ mainly on the smaller non-expert remainder. That explains why UD-Q4_K_XL is 155.095GB and UD-Q8_K_XL is 161.870GB. The labels describe mixed tensor recipes, not uniform bits per parameter.

Unsloth’s published Wikitext-2, context-512 proxy reports Q8 as effectively identical to its converted target reference and Q4 as close. Those developer measurements are useful conversion evidence, but perplexity, KL divergence and top-token agreement are not substitutes for compiled patches, schema validation, correct tool sequences or retrieval at long context. Kingy did not recover scored Q4 or Q8 rows, so those developer claims remain outside this ranking.

Which quant should you download?

Choose UD-IQ2_M when the measured pair is your decision

IQ2 is the evidence-led default from this run. It saved exactly 13,280,919,744 bytes versus IQ3, decoded slightly faster and gave up only two decision points. That is a better measured size/behavior trade-off than paying 14.6% more download size for a 0.47-point percentage difference.

Choose UD-IQ3_XXS only for a specific reason

IQ3 is defensible if its three extra tool-call points matter to your workload and the extra 13.3GB is irrelevant. Validate that hypothesis with your own tools, schemas and multi-turn traces. This run does not show a broad IQ3 quality tier, and it showed no coding-patch advantage.

Treat IQ1, Q4 and Q8 as unranked

IQ1 is still the smallest capacity candidate, Q4 remains the near-Q8-size alternative and Q8 carries the strongest developer conversion-fidelity claim. But the local evidence cannot tell you how their task behavior compares. Do not place them above or below IQ2/IQ3 using the missing rows.

The public evidence appendix preserves all 354 row-level scores, corruption checks, performance records, visible final model outputs and file hashes. It deliberately excludes hidden answer-key material and private reasoning text.

Reproducibility details

• GGUF repository: unsloth/DeepSeek-V4-Flash-0731-GGUF at 109848da….
• Official model reference: deepseek-ai/DeepSeek-V4-Flash-0731 at 7872f01b….
• Runtime: llama.cpp b10219, commit c629da565c80b0b17fac6262acdca4d772e745d8.
• Inference: split-mode layer, all layers requested for GPU offload, 32,768-token allocation, flash attention on, F16 K/V cache, batch and micro-batch 2,048, 52 threads, context shift off, warmup disabled.
• Sampling: temperature 1.0; top-p 0.95 for agentic tasks and 1.0 otherwise; min-p 0; top-k 0; seeds 731, 1731 and 2731; reasoning effort high.
• Plan ID: cd6b4c0a2a9329ff395278dcc2733641212a0a70d6d9dca3660bb670f734f760.
• Tokenizer fingerprint: 48a40b21662bbb768d0231a919d54babdce0ad90f70a9361b278cfe51d529b5a.

The 12 shards used for the complete IQ2 and IQ3 runs and the Q8 smoke were SHA-256 verified on the benchmark host before loading. The local emergency pull then rechecked every manifest-listed evidence/result file: 3,162 for IQ2, 3,164 for IQ3 and 26 for the Q8 smoke. The appendix identifies those manifests and their hashes.

Limitations

• This is a complete two-quant comparison, not the planned five-quant comparison. IQ1, Q4 and Q8 quality scoring is unavailable and excluded.
• Long-context retrieval was NOT RUN at every rung, including 32K. Maximum stable context and the 128K/384K/1,048,576-token ladder were not tested.
• No native-model or official-endpoint reference ran on the same task suite.
• No page-cache-controlled cold/warm load experiment was performed.
• The other eight repository labels and 96GB/128GB hardware classes were not empirically tested.
• The multilingual scorer is a literal-marker diagnostic. Multilingual factual preservation, multilingual degradation and general translation fluency are NOT ESTABLISHED.
• The manual IQ3 coherence control used a different request path and settings from the scored suite. It ruled out one immediate corruption signature only; it was not an exact scored-path validation.
• Results belong to this quant×runtime×hardware combination. Runtime kernels, templates, parsers and sampling can change behavior.

Final verdict

Download UD-IQ2_M if you want the best supported size/behavior trade-off from the evidence Kingy actually completed. It was 13.281GB smaller, slightly faster and effectively tied with IQ3 on the decision rubric. Download IQ3 only when you can name a workload-specific reason—most plausibly its small tool-call advantage—and can validate that advantage yourself.

Do not convert missing data into a five-quant ranking. IQ1 may be the only payload that fits your machine; Q8 may be the best conversion-fidelity target; Q4 may fit where Q8 does not. None of those capacity or developer-proxy arguments is a Kingy task-quality result.

The useful conclusion is narrower and stronger: between the two measured downloads, more bytes did not buy a decisive behavior gain.

FAQ

What is the best tested DeepSeek V4 Flash 0731 quant?

Among the two quants with complete locally verified scored runs, UD-IQ2_M is the practical default. IQ3 led the rubric by only two points out of 426 while requiring 13.281GB more payload.

Did Kingy test five quants?

No. The plan covered five, but publishable scored evidence is complete only for IQ2 and IQ3. Q8 completed a load-only smoke. IQ1, Q4 and Q8 are unranked for quality.

Was IQ3 better at tool use?

It scored 234/300 versus IQ2’s 231/300. That three-point edge was partly offset by IQ2’s one-point reasoning edge. Validate tool behavior on your schemas before treating the difference as durable.

Which one was faster?

IQ2’s median decode rate was 34.89 tokens/s versus 34.17 for IQ3 on the same six-A40 host. The run was not a page-cache-controlled performance benchmark, so treat the roughly 2% difference as an observed result, not a universal speed ratio.

Can Q8 run at 32K context?

The Q8 payload loaded in a generation-free smoke with a 32K allocation. No generation or retrieval ran, so stable 32K inference was not established.

Where are the row-level results?

Download the versioned evidence appendix. It contains 354 row records, visible final outputs, scores, performance, corruption checks and cryptographic hashes, with hidden keys and private reasoning excluded.

Sources

• DeepSeek official repository, pinned revision: model card, configuration, encoding and weights.
• Unsloth GGUF repository, pinned revision: exact representative files and metadata.
• Revision-pinned Hugging Face API manifest: file sizes and repository LFS SHA-256 identifiers.
• Unsloth DeepSeek V4 documentation: developer quantization analysis, proxy metrics and memory guidance.
• DeepSeek V4 technical report: architecture and long-context design.