Which Qwen3.8-27B GGUF Should You Download? Q2–Q8 Quantization Guide

Right now

Trending on Kingy

Keep reading with the stories getting the most attention now.

1. 01 Blog · Aug 14

   Qwen3.8-27B Runs on a Laptop—But Here’s the Hardware You Actually Need

   Read story →
2. 02 Blog · Aug 17

   Qwen3.8-27B vs Qwen3.6-27B vs Gemma 4 31B: Which Is Best for a 24GB GPU?

   Read story →
3. 03 Blog · Aug 10

   America’s Answer to Qwen? Muse Glimmer Wins the 24GB Mac Fit Test

   Read story →

Verdict: Most people with 24 GB of VRAM should start with Unsloth’s exact Qwen3.8-27B-UD-Q4_K_XL.gguf, not an unnamed “Q4.” It is 17.92 decimal GB (16.69 GiB), leaves materially more context headroom than Q5 or Q6, and scored close to larger files in the best disclosed same-test quantization comparison available at this article’s cutoff. On 16 GB, AtomicChat’s 13.84 GB Qwen3.8-27B-AD-IQ3_S.gguf is the more realistic fully resident choice. On 32 GB, LM Studio’s 22.43 GB Qwen3.8-27B-Q6_K.gguf offers a strong high-fidelity/context balance. At 48 GB or more, Q8 becomes rational. It is rarely the best use of a constrained 32 GB device.

Kingy.ai did not download roughly half a terabyte of candidate files and run the controlled matrix described in our editorial brief. This is a source-audited file-selection guide. Quality figures below are publisher-reported common-test measurements against the original BF16 weights; file sizes, revisions and checksums come from Hugging Face manifests; memory figures are calculated lower bounds. Those evidence classes stay separate throughout.

This guide targets Qwen3.8-27B, the 27-billion-parameter vision-language model released in August 2026. The older Qwen3-8B and Qwen3.8-Max are different models. If you are still deciding what hardware can run Qwen3.8-27B, use our hardware requirements guide. This guide starts after that decision: you already have the machine and need the exact file.

The one-minute Qwen3.8-27B download chooser

These are starting points, not promises that a nominal memory number guarantees a stable context. The table does not include the operating system, runtime graph, allocator slack, recurrent state, KV cache, vision projector or parallel slots. It also does not call a publisher’s file “Kingy-tested.”

Why “download Q4” is not a complete answer

Q4_K_M looks like a precise label. It is not a complete identity.

The label tells you the quantization family the converter requested. It does not tell you which source revision was converted, which llama.cpp build performed the conversion, whether an importance matrix influenced tensor allocation, whether embeddings and the output head were kept at higher precision, whether the model’s MTP head was embedded or split into another file, or whether a vision projector is present.

Current repository manifests show the difference. At the cutoff, files carrying Q4_K_M in their names ranged from 16.81 GB in the LM Studio repository to 18.97 GB in ggml-org’s repository. Bartowski’s was 17.77 GB; Unsloth’s conventional version was 17.11 GB. The files are all quantizations of the same official base model, but their bytes and tensor treatment are not interchangeable.

AtomicChat’s publisher-authored common-test comparison reported KL divergence of 0.02094 for LM Studio’s 16.8 GB file, 0.01470 for ggml-org’s 19.0 GB file and 0.01126 for its own 17.1 GB architecture-aware file. Lower is better. AtomicChat sells a related app and authored the comparison, so those values are publisher-reported rather than independent validation. Its raw logs and BF16-relative method are public, which makes the claim auditable.

This is why repository, revision and filename belong in every recommendation. “Q4” alone is not reproducible.

The Qwen3.8-27B files that matter

The canonical checkpoint is Qwen/Qwen3.8-27B revision 1d4bf0f. Its model card identifies a dense 27B vision-language model under Apache 2.0, with 64 language layers, 262,144-token native context, optional extension to one million, thinking controls and multi-token prediction. The 18 safetensor shards total 55.56 decimal GB. That is the reference model; every GGUF here is a derived artifact.

Five repositories deserve attention for different reasons:

• Unsloth offers the broadest easy-to-navigate ladder, including conventional and Dynamic V3.0 files from roughly 9 GB through 31.5 GB. Its card labelled Dynamic V3.0 as a preview. That makes the files useful candidates, not immutable truth.
• LM Studio community offers a compact Q4/Q6/Q8 set converted with disclosed llama.cpp release b10430. It is a sensible compatibility-oriented lane for LM Studio and llama.cpp users.
• ggml-org is the upstream runtime-project repository. It includes BF16, Q4 and Q8 language files plus separate vision and MTP artifacts. Its sparse card does not provide a quality study.
• Bartowski publishes a broad imatrix ladder, names the conversion release and discloses a calibration corpus rendered through this model’s chat template. The recommendations on that card remain publisher opinion.
• AtomicChat publishes architecture-aware files, calibration data, raw KLD logs and cross-publisher results. That is unusually useful evidence, but it also has a commercial interest in its own artifacts.

Download counts answer “what is popular,” not “what is better.” They are omitted from the ranking.

Companion files are part of the decision

Text-only use needs only the language GGUF. Image input normally needs a separate multimodal projector. Unsloth’s pinned mmproj-F16.gguf is 0.93 GB (0.86 GiB). Bartowski and AtomicChat publish similarly sized F16 projectors, while ggml-org also offers a smaller Q8 projector. Do not assume a front end fetched the right projector merely because it accepted the language file.

MTP treatment also varies. Bartowski and AtomicChat say the MTP head is embedded in their files. ggml-org publishes separate MTP GGUFs ranging from 1.68 GB to 5.95 GB. A command copied between repositories can therefore be wrong even when the base model name looks identical.

How the available evidence was normalized

Kingy.ai did not merge unlike evidence into one “intelligence score.” We used five labels:

1. Official model/runtime documentation for architecture, context, sampling, runtime flags and repository identity.
2. Publisher common-test for BF16-relative KL divergence and top-1 agreement published by AtomicChat.
3. Publisher documentation for conversion builds, imatrix corpora and MTP/projector treatment.
4. Calculated estimate for GiB conversion and full-attention KV-cache arithmetic.
5. Not verified where no controlled evidence supports a claim.

AtomicChat’s quantization study used the original BF16 GGUF as the reference, 87 held-out chunks at 4,096 context, per-token KL divergence and top-1 next-token agreement. It says the held-out text was excluded from calibration and publishes its logs. This is much stronger than comparing screenshots or one coding answer. A 0.002 KL difference still does not prove that your agent completes more tasks, and the study does not establish a universal speed ranking.

We use the common-test results to identify the compression curve, then make memory-tier recommendations using actual bytes and conservative headroom. CUDA speed stays separate from Metal, Vulkan and CPU inference. Vision results stay separate from text quality. MTP belongs to decoding, so it stays outside the weight-accuracy ranking.

Q2 versus Q3 versus Q4 versus Q5 versus Q6 versus Q8

The quality curve becomes steep below roughly 14 GB and flat above roughly 22 GB.

Q2 is a capacity hack

The 11.14 GB Atomic IQ2 file disagreed with the BF16 reference’s top next token about 12.8% of the time in the publisher’s held-out corpus. That is not a direct task-failure rate, but it is a loud warning. Use Q2 to prove the model can run, to tolerate partial offload, or to work within a hard device limit—not because “27B at two bits” is automatically better than a smaller model at sane precision.

Q3 is the realistic 16 GB lane

Moving from Atomic’s IQ2_S to IQ3_S costs 2.70 GB and cuts reported KL divergence by about two-thirds. That is the largest practical improvement in this shortlist. On a 16 GB accelerator, 13.84 GB also leaves more than 2 GB for cache and runtime, whereas a 16–18 GB Q4 file does not fit once overhead arrives.

Q4 is the default because it preserves headroom

Unsloth’s 17.92 GB Dynamic Q4 lands at 96.02% top-1 agreement in the disclosed common test. The 18.55 GB Atomic mixed Q5/Q4 file improves that to 96.43% for only 0.63 GB more. The gain is real in the publisher’s metric, but it does not show that every practical task improves. On 24 GB, take Unsloth Q4 for more context margin and the Atomic mixed file for short-context quality.

Q5 is the diminishing-returns hinge

At 20.22 GB, Unsloth’s Dynamic Q5 roughly halves the reported divergence of its Dynamic Q4. If your prompts are short and the whole model still fits, that is attractive. If you need 64K context, vision, multiple slots or a desktop full of other applications, the extra 2.30 GB can be more valuable as headroom.

Q6 is the high-fidelity sweet spot on 32 GB

LM Studio’s 22.43 GB Q6_K reached 97.93% top-1 agreement with the BF16 reference in the publisher common test. Atomic’s 25.01 GB architecture-aware Q6 reached 98.67%, but consumes another 2.57 GB. The smaller Q6 is the better default when context matters; the larger one is the quality-first option.

Q8 still diverges from BF16

Atomic’s Q8_0 still produced nonzero divergence and 98.93% top-1 agreement. Unsloth’s 31.46 GB Dynamic Q8_K_XL improved the metric slightly, but the 2.57 GB size increase buys only 0.15 percentage points of top-1 agreement in this corpus. On a 32 GB card, either file crowds out cache and runtime. On 48–64 GB, Q8 makes more sense.

Which quant fits after context is included?

The model file is only part of the memory bill. Qwen’s official configuration has 16 full-attention layers, four KV heads and head dimension 256. Conventional BF16/F16 attention-cache arithmetic is:

16 layers × 4 KV heads × 256 dimensions × 2 (K and V) × 2 bytes

That equals 65,536 bytes per token for the full-attention component. The result is a lower bound, not total loaded memory:

Quantized cache has format overhead. The table also excludes Gated DeltaNet recurrent state, graph buffers, activations, allocator slack, projector memory, operating-system reservation and batching. Successful allocation is not evidence that long-context retrieval remains accurate.

A 28.89 GB Q8 file is a poor 32 GB recommendation despite “fitting” on paper. It leaves roughly 5 GiB before the runtime starts. A 17.92 GB Q4 file leaves about 7.3 GiB on a nominal 24 GiB card, enough to make 16K or 32K a plausible starting point. For the broader capacity picture, see the full Qwen3.8-27B memory ladder and why the 262K context window changes memory needs.

Best Qwen3.8-27B GGUF by hardware and workload

Below 12 GiB: an emergency file, not a default

AtomicChat’s AD-IQ2_S is the least-bad measured option in this guide below 12 decimal GB. It is appropriate when loading the 27B model matters more than retaining the last measure of fidelity. Start text-only at 4K or 8K. If a smaller well-quantized model completes your work more reliably, use the smaller model.

16 GB: choose Q3 and stay fully resident

AtomicChat’s AD-IQ3_S is 12.89 GiB. That leaves enough nominal space for an 8K cache and runtime overhead on a 16 GiB accelerator. A Q4 file with CPU offload may produce better answers but slower tokens; that becomes a latency trade, not a clean 16 GB fit. For a coding agent, staying fully resident is often worth more than squeezing in a larger file that pauses between every token.

24 GB: Q4 for context, mixed Q5/Q4 for short prompts

Unsloth’s UD-Q4_K_XL is the general recommendation. Start at 16K and increase only after observing peak memory. If your workload is short-context code generation or structured reasoning and 16K is enough, AtomicChat’s 18.55 GB AD-Q5_K_M-Q4_K_M has better publisher-reported BF16 retention for only 0.58 GiB more.

Do not begin with Q6 merely because a 22–23 GB file is smaller than “24 GB.” Runtime and cache have nowhere sensible to live.

32 GB: Q6 is finally comfortable

LM Studio’s 22.43 GB Q6_K leaves about 9.1 GiB of nominal space on a 32 GiB device. That is the cleanest high-fidelity/context balance in the current measured set. AtomicChat’s 25.01 GB Q6 is defensible for short-context quality, but the smaller LM Studio file is the safer all-purpose start.

48 GB and 64 GB: Q8 becomes reasonable

AtomicChat’s 28.89 GB Q8_0 leaves substantial room for cache at 48 GB and comfortable room at 64 GB. It is the high-fidelity choice for repeated evaluation, vision work or tasks where small logit changes matter. It is not automatically faster, and it is not BF16.

Apple unified memory: capacity is shared

On a 24 GB Mac, use the 16 GB recommendation unless you close other memory-heavy applications and accept limited context. On 32 GB, Unsloth Dynamic Q4 is the safer GGUF default. On 48/64 GB, LM Studio Q6_K or Q8 is practical through llama.cpp. If you are committed to Apple Silicon and do not need GGUF portability, LM Studio’s pinned 4-bit MLX conversion deserves a separate test; this source audit does not claim that its speed or quality is directly comparable to the GGUF rows.

MTP, reasoning effort and cache settings change the result

MTP can accelerate generation by using the model’s trained multi-token-prediction heads as a draft. Current llama.cpp documents --spec-type draft-mtp. It does not promise a fixed speedup: acceptance varies with output entropy, backend and configuration. Code completion may accept more draft tokens than exploratory prose.

Qwen’s official model card recommends temperature=1.0, top_p=0.95, top_k=20 and min_p=0.0 for thinking; non-thinking uses temperature=0.7, top_p=0.8, top_k=20 and min_p=0.0. Thinking is enabled by default. Supported reasoning_effort values are low, medium and xhigh, with xhigh the default in the official chat template.

Reasoning effort can dominate wall time and output length. A Q6 run at xhigh is not a fair speed comparison with Q4 at low. Likewise, preserved thinking in an agent conversation changes the context contents. Record preserve_thinking, sampling and reasoning effort before blaming the quant.

For a first accuracy run, leave MTP off. Once the file and prompt are stable, enable it and record accepted draft tokens, generation speed and memory. That isolates serving mechanics from weight quality.

Do you need the vision projector?

Not for text. You do need a compatible projector for images or video through llama.cpp’s multimodal path. The runtime’s pinned multimodal documentation explains that image encoding lives in a separate mmproj model and typically requires two GGUF files.

Unsloth’s pinned mmproj-F16.gguf is 0.93 GB (0.86 GiB) with Hugging Face linked checksum cbb841a9ee0636b2ec172f5bb8df2ea8dfeb01e90fe7c6126581d662a0b4e43e. Add that to storage and memory planning. Do not include it in text-only comparisons, and do not treat one handwritten-image demonstration as proof of general vision retention.

Because different repositories name the projector differently, keep the language file and projector from a documented compatible pair. If your frontend auto-downloads one, verify the resolved repository and revision.

GGUF versus MLX, FP8 and NVFP4

GGUF is the broadest interoperability choice. Hardware-native formats can still fit some systems better.

The official FP8 checkpoint was pinned at 017b9c7. Unsloth’s NVFP4 repository had already advanced to 7d6f8d4 by the cutoff, a reminder that launch-week optimized formats are moving targets. Blackwell users should start with current runtime documentation, not a stale command copied from a launch thread.

Choose GGUF when portability, mixed CPU/GPU offload or llama.cpp tooling matters. Choose MLX when you are all-in on Apple and can validate the exact conversion. Choose FP8 or NVFP4 when your serving stack and accelerator explicitly support them.

Download and verification checklist

1. Open the immutable repository-revision link, not only /main.
2. Match the full filename, including UD-, AD-, IQ, K_M, K_XL and any mixed suffix.
3. Check the byte size before and after download.
4. Record the Hugging Face linked checksum. For Unsloth UD-Q4_K_XL, it is bee238bbeb3dc0a34bde4d0dedbaee1f98c009e8bb4226f03070054c12fb1372 at revision f1bfb127.
5. Use a llama.cpp build at least as new as the publisher’s conversion build; Qwen3.8 needs recent qwen35 architecture support.
6. Confirm whether MTP is embedded or separate.
7. Download a compatible projector only for multimodal use.
8. Start at 8K or 16K, watch actual peak memory and increase context gradually.
9. Preserve the official chat template and record thinking, sampling and cache settings.

For installation rather than file choice, follow our Qwen3.8-27B local run instructions.

Final recommendation

Use Unsloth UD-Q4_K_XL on 24 GB as the broad default. It sits near the useful bend in the publisher-reported compression curve and preserves room for context. Atomic AD-IQ3_S is the 16 GB compromise. LM Studio Q6_K is the 32 GB high-fidelity default. Atomic Q8_0 belongs on 48–64 GB systems, not on a 32 GB card that still needs a cache.

Those choices can change if a repository replaces an artifact, llama.cpp fixes conversion behavior, an independent same-test study contradicts the current publisher results, or a new hardware-native format becomes stable. Keep the bytes, revision and evidence label with the filename.

FAQ

Which Qwen3.8-27B GGUF should most people download?

For a 24 GB accelerator, start with unsloth/Qwen3.8-27B-GGUF revision f1bfb127, file Qwen3.8-27B-UD-Q4_K_XL.gguf. It is 17.92 GB and leaves more context headroom than Q5 or Q6. Unsloth labelled Dynamic V3.0 preview, so keep the revision pinned.

What is the best Qwen3.8-27B quantization for 16 GB VRAM?

AtomicChat’s Qwen3.8-27B-AD-IQ3_S.gguf is the source-audited starting choice. Its 12.89 GiB file leaves plausible room for an 8K cache and runtime. A Q4 can run with CPU offload, but it is no longer a clean fully resident 16 GB configuration.

What is the best Qwen3.8-27B quantization for 24 GB VRAM?

Use Unsloth UD-Q4_K_XL for general work and context headroom. For short prompts where quality matters more than context, AtomicChat’s 18.55 GB mixed AD-Q5_K_M-Q4_K_M showed better BF16 retention in the publisher common test.

Is Q5 noticeably better than Q4?

In the disclosed common test, Unsloth Dynamic Q5 cut KL divergence from 0.00955 to 0.00437 and raised top-1 agreement from 96.02% to 97.28%. That is measurable distributional retention, not proof that every task will feel different. Q5 is most attractive when its extra 2.30 GB does not steal needed context.

Is Q8 worth the extra memory?

On 48–64 GB, sometimes. On 32 GB, usually not. Atomic Q8_0 was close to BF16 at 98.93% top-1 agreement, but its 26.90 GiB file leaves too little room for a large cache and runtime on a 32 GiB card.

What is the difference between Q4_K_M and UD-Q4_K_XL?

Q4_K_M is a conventional K-quant recipe label. Unsloth’s UD-Q4_K_XL is a Dynamic V3.0 mixed-precision artifact that allocates precision differently and is larger. The exact implementation, source revision and per-tensor choices matter more than the visible “4.”

Which Qwen3.8-27B GGUF publisher is best?

No publisher wins every criterion. Unsloth has the broadest ladder; LM Studio has a small compatibility-oriented set; ggml-org is the upstream conversion source; Bartowski discloses a useful imatrix process; AtomicChat publishes unusually detailed common-test evidence and raw logs but has a commercial interest. Choose an exact artifact, not a publisher badge.

Do I need the mmproj file for text-only use?

No. The projector is for image or video encoding. Text-only inference needs the language GGUF. For multimodal use, add a compatible projector and include roughly 0.86 GiB for an F16 file in storage and memory planning.

Does MTP change answer quality?

MTP is a decoding mechanism; the stored language weights stay unchanged. In a correct speculative-decoding implementation it should preserve the target distribution, but acceptance, speed and memory vary. Establish an MTP-off baseline, then enable --spec-type draft-mtp and measure your workload.

Can a GGUF use the full 262K context?

The model supports 262,144 tokens natively, but practical allocation depends on quant, KV type, recurrent state, buffers, projector and hardware. BF16 full-attention KV arithmetic alone is 16 GiB at 262K. A successful allocation also does not prove reliable retrieval at that length.

Is NVFP4 better than GGUF on an RTX 5090?

NVFP4 can be the faster hardware-specific serving path on Blackwell when the current runtime supports the exact checkpoint. That does not make it universally higher quality or easier to deploy. Compare it in a separate runtime lane; do not paste its speed beside a Metal or CPU GGUF result.

Is Qwen3.8-27B the same model as Qwen3-8B?

No. Qwen3.8-27B is a 27B dense vision-language model released in August 2026. Qwen3-8B is an older 8B model with different weights, architecture and artifacts. A missing decimal point can send you to the wrong repository.

Sources, methodology and change log

Known limitations: Kingy.ai did not run the files. The decisive quality evidence is publisher-authored, not independent; KL divergence and top-1 agreement do not replace task-level evaluation; loaded-memory and context totals were not measured on a common rig; speed was not ranked; repository heads can change after the pinned revisions.

Change log: August 16, 2026: first source-audited edition. Repository heads, artifact bytes, linked checksums, runtime documentation and competing format revisions refreshed at 19:22 PDT.