Qwen3.8-Flash-Next VRAM & vLLM Hardware Guide

Kingy verdict, verified August 28, 2026: The official Qwen3.8-Flash-Next FP8 checkpoint is a multi-GPU deployment, not a 6B model that fits on a gaming card. Its 131 safetensor shards total 185.523 GB, or 172.782 GiB, before vLLM allocates caches, workspaces or CUDA graphs. The current vLLM recipe budgets 250 GB of aggregate VRAM for FP8 and validates two GB300 GPUs as the minimum GB300 layout, with four GB300 GPUs recommended. On Hopper, the documented FP8 path is eight H200 GPUs with tensor plus expert parallelism (TEP8). Four MI355X GPUs are the documented AMD path.

Do not buy hardware from the “6B active parameters” line. Qwen’s official model card lists a 125B main model with 6B activated per token, plus 51B N-gram embeddings and a 4B multi-token-prediction module. The complete package is why Hugging Face can display about 180B parameters while vLLM’s architecture summary says 176B: vLLM’s number excludes the 4B MTP module.

This is also not a released model named “Qwen4 Preview.” Qwen’s launch post calls Qwen3.8-Flash-Next an experimental preview of the architecture intended to underpin Qwen4. The checkpoint you can download is still Qwen/Qwen3.8-Flash-Next or its official FP8 counterpart.

Testing disclosure: Kingy inspected the official launch, model cards and pinned repository metadata; the Qwen technical report; the current vLLM recipe and image digests; hardware specifications; and active upstream implementation issues. We did not run the model. Our available M4 Pro MacBook has 24 GB of unified memory and cannot hold either official checkpoint. Configurations below are labelled officially validated, calculated, community-reported or not officially validated.

Qwen3.8-Flash-Next hardware requirements at a glance

Checkpoint and layout	Weight files	Published runtime budget	Host RAM	Context status	Evidence
FP8, 2x GB300	172.782 GiB total	250 GB aggregate VRAM in recipe; TP2 minimum on GB300	51 GB plus runtime headroom if PLE offload is enabled	262,144 advertised; full-length single request not tested by recipe	Officially validated minimum
FP8, 4x GB300	172.782 GiB total	4x 288 GB HBM3e hardware; TP4 or TEP4	Same offload floor; provision more for the OS and loader	Native limit configured in validation	Officially recommended
BF16, 2x GB300	335.276 GiB total	423 GB aggregate recipe budget; about 190 GiB used per GPU in TP2 validation	No complete published host-RAM result	Native limit configured in validation	Officially validated
FP8, 8x H200	172.782 GiB total	8x 141 GB HBM3e; use TEP8, not plain TP8	Platform-dependent	Native limit configured in validation	Officially validated
FP8, 4x MI355X	172.782 GiB total	4x 288 GB HBM3e	Platform-dependent; PLE CPU offload is documented as NVIDIA-only	`--max-model-len auto` in recipe	Officially validated recipe
FP8, 4x RTX PRO 6000 Blackwell	172.782 GiB total	384 GB decimal VRAM total	No published peak host-RAM result	Profile uses reduced concurrency	Marked verified in recipe metadata; detailed measurements unpublished
RTX 5090, RTX 4090, Apple M-series	172.782 GiB FP8; 335.276 GiB BF16	Far below official checkpoint weight size	CPU/offload does not create vLLM support for these paths	Not applicable	Does not fit the official checkpoints; no official vLLM validation

The table deliberately separates file size from runtime memory. A 172.782 GiB checkpoint does not “need 173 GB of VRAM.” It needs that much storage for the tensors if all of them remain on accelerators, plus memory for model state, recurrent and attention caches, kernels, communication buffers, compilation and request concurrency. vLLM’s current recipe expresses that operational difference as a 250 GB FP8 minimum.

Why normal VRAM calculators get this model wrong

Qwen3.8-Flash-Next combines three memory stories that a parameter slider cannot represent.

First, it is an ultra-sparse mixture of experts. Only 6B main-model parameters activate for each token, which reduces matrix multiplication. It does not mean the other experts disappear from memory. The server still needs access to the resident or sharded weights.

Second, Qwen adds a 51B-parameter N-gram embedding table, called PLE in the vLLM implementation. It behaves more like a large lookup memory than another stack of transformer layers. Qwen places it near the beginning of the network and can prefetch selected rows from host memory. That lowers accelerator residency when offload works, but shifts capacity and bandwidth requirements to the CPU side.

Third, the published checkpoint includes a 4B MTP module for speculative decoding. Qwen’s model card therefore describes 125B main-model parameters, 51B N-gram parameters and 4B MTP parameters. The useful numbers are:

Number	What it means	What it does not mean
6B	Main-model parameters activated per generated token	Weight memory or download size
125B	Main language-model parameter count	Entire packaged checkpoint
51B	Additional N-gram/PLE embedding capacity	Extra dense compute on every token
4B	MTP module packaged for speculative decoding	Always-enabled runtime overhead
About 180B	125B + 51B + 4B package-level explanation	A single precision or runtime-memory requirement

This architecture is exactly why the raw files matter more than a generic formula.

Exact BF16 and FP8 checkpoints

The two official repositories were pinned at the following revisions for this guide:

Precision	Exact model ID	Pinned revision	Safetensor files	Exact safetensor bytes	Human-readable size
BF16	`Qwen/Qwen3.8-Flash-Next`	`de4b8e4d43b917e7706784d8bb445c9af86a3540`	131	360,000,192,888	360.000 GB / 335.276 GiB
FP8	`Qwen/Qwen3.8-Flash-Next-FP8`	`970c569adaca6b35532111fd6b27351b2baefe50`	131	185,523,317,458	185.523 GB / 172.782 GiB

The FP8 configuration declares dynamic FP8 quantization with 128 x 128 weight blocks. It leaves a long list of modules unconverted, so “one byte per parameter” is not an exact size calculation. The repository bytes above are exact.

Qwen says the FP8 checkpoint’s reported performance is nearly identical to the original model. That is a vendor statement, not an independent Kingy quality test. We found no same-test, independently reproduced BF16-versus-FP8 quality result at this evidence cutoff.

For storage, plan beyond the tensor total. A practical floor is roughly 220 GB free for the FP8 repository and 400 GB for BF16, with more if you retain revisions, Docker layers or a second copy. These are provisioning recommendations, not measured minimums.

The vLLM support state is still a preview

Use the dedicated image from the Qwen3.8-Flash-Next recipe:

NVIDIA: vllm/vllm-openai:qwen38-flash-next
ROCm:   vllm/vllm-openai-rocm:qwen38-flash-next

At the cutoff, the vLLM recipe says 0.29.0+, marks a nightly build as required and explicitly says that PyPI installation is not supported for this recipe. The model-support pull request, vllm-project/vllm#53896, remained open. Hugging Face’s generic interface simultaneously displayed pip install vllm; do not treat that generated snippet as proof that the released PyPI wheel contains this preview implementation.

The reviewed immutable image digests were:

NVIDIA multi-architecture image:
sha256:fc120ece0a388cc0aa1caad4a9f1cd92113484ab7ec2fd0efadd62585be05bf8

ROCm image:
sha256:a3084fee2360c54d85f69d5d6edcb8a8ce308a6d68b7fd4dbb9eea6a14bfd156

The NVIDIA image’s own OCI labels report its build commit as unknown. Pinning the digest preserves the bytes, but it does not provide a trustworthy source-commit mapping. Record the output of vllm --version and the startup banner in every real deployment.

Official FP8 launch on four GB300 GPUs

The following wraps the recipe’s serving command in vLLM’s documented Docker pattern. It pins both the image and model revision. Replace the cache mount with a real path that has at least 220 GB free.

docker run --rm --gpus all \
  --ipc=host \
  -p 8000:8000 \
  -v /path/to/hf-cache:/root/.cache/huggingface \
  -v qwen38-vllm-cache:/root/.cache/vllm \
  --entrypoint /bin/bash \
  vllm/vllm-openai@sha256:fc120ece0a388cc0aa1caad4a9f1cd92113484ab7ec2fd0efadd62585be05bf8 \
  -lc 'exec vllm serve Qwen/Qwen3.8-Flash-Next-FP8 \
    --revision 970c569adaca6b35532111fd6b27351b2baefe50 \
    --tensor-parallel-size 4 \
    --gpu-memory-utilization 0.90 \
    --max-num-seqs 256 \
    --enable-prefix-caching \
    --no-enable-flashinfer-autotune \
    --enable-auto-tool-choice \
    --tool-call-parser qwen3_xml \
    --reasoning-parser qwen3'

The recipe calls TP2 the minimum validated FP8 deployment on GB300. TP4 and TEP4 are the full-tray configurations it recommends. Add --enable-expert-parallel for TEP4.

The command above is source-audited, not Kingy-executed. The exact image digest, checkpoint revision and flags were checked; no GB300 host was available here.

Official H200 path: TEP8, not plain TP8

The FP8 checkpoint uses 128-wide quantization blocks. The recipe warns that plain TP8 is incompatible with that block layout and specifies TEP8 with the Triton MoE backend:

vllm serve Qwen/Qwen3.8-Flash-Next-FP8 \
  --revision 970c569adaca6b35532111fd6b27351b2baefe50 \
  --tensor-parallel-size 8 \
  --enable-expert-parallel \
  --moe-backend triton \
  --gpu-memory-utilization 0.85 \
  --max-num-seqs 256 \
  --enable-prefix-caching \
  --no-enable-flashinfer-autotune \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3

TEP8 still sets tensor parallelism to eight. Enabling expert parallelism changes how MoE experts are distributed and executed. On this checkpoint, “I have eight GPUs, so TP8 is fine” is the wrong shortcut.

Official AMD path: four MI355X GPUs

AMD lists 288 GB of HBM3e per MI355X. The vLLM recipe provides this four-GPU ROCm configuration:

export VLLM_ROCM_USE_AITER=1
export VLLM_ROCM_USE_AITER_MOE=0

vllm serve Qwen/Qwen3.8-Flash-Next-FP8 \
  --revision 970c569adaca6b35532111fd6b27351b2baefe50 \
  --tensor-parallel-size 4 \
  --max-model-len auto \
  --gpu-memory-utilization 0.9

Do not copy the NVIDIA PLE-offload claim into this path. The recipe currently says N-gram embedding offload runs only on NVIDIA devices. Upstream ROCm reports also describe active MTP and cache issues on preview code, so pin the image and test your exact request pattern before production use.

N-gram/PLE CPU offload

The initial offload implementation keeps the large N-gram lookup memory in host RAM and asynchronously prefetches selected rows. The official recipe documents this environment variable:

VLLM_PLE_CPU_OFFLOAD=1 \
vllm serve Qwen/Qwen3.8-Flash-Next-FP8 \
  --revision 970c569adaca6b35532111fd6b27351b2baefe50 \
  --tensor-parallel-size 4 \
  --max-model-len 262144 \
  --no-enable-flashinfer-autotune \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3

The documented host-memory floor is at least 51 GB plus runtime headroom. That is a component floor, not a complete server-RAM recommendation. For an FP8 offload deployment, 128 GB of system RAM is a sensible planning floor, and more is appropriate for high concurrency, model download staging and monitoring. That 128 GB figure is Kingy’s provisioning judgment; it is not an official measured minimum.

PLE offload is optional for TP and TEP but required for vLLM’s disaggregated expert-parallel strategy. Pipeline parallelism is not supported by the initial N-gram implementation.

Do not assume offload makes a single small GPU viable. An open vLLM report reproduced a TP1 warm-up deadlock on a GB10 system with a third-party NVFP4 checkpoint. Another open issue reports an A100 failure on the official FP8 model. Those are community reports with specific code and hardware, not universal proof, but they remove any basis for recommending TP1 or A100 today.

Native 262K context and the 1M YaRN extension

The checkpoint advertises 262,144 native tokens. If --max-model-len is omitted, vLLM reads that value from the model configuration. The recipe says startup and bounded evaluation were validated with the native limit configured, but a single full 262K-token request was not tested.

For one million tokens, vLLM documents an explicit static YaRN override:

VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 \
vllm serve Qwen/Qwen3.8-Flash-Next-FP8 \
  --revision 970c569adaca6b35532111fd6b27351b2baefe50 \
  --tensor-parallel-size 4 \
  --hf-overrides '{"rope_parameters":{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":262144}}' \
  --max-model-len 1000000 \
  --no-enable-flashinfer-autotune \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3

Test short-context quality before making YaRN the default. A configured maximum is not a measured one-million-token service-level result. Request length, batch size, prefix-cache reuse, multimodal tokens and concurrency all change runtime memory.

MTP speculative decoding

Qwen packages a trained MTP module. vLLM enables three speculative tokens with:

--speculative-config '{"method":"mtp","num_speculative_tokens":3}'

MTP can improve decode throughput when the proposed tokens are accepted, but it is not free. It adds model state and temporary memory, and the recipe’s troubleshooting advice is to reduce the speculative-token count below three when memory pressure appears. Measure acceptance length and end-to-end throughput on your workload. Tokens per second without acceptance data is an incomplete MTP result.

Can your GPU run it?

Hardware	Memory	Official FP8 checkpoint verdict
RTX 4090	24 GB	No. Far below weight size; no official path.
RTX 5090	32 GB	No. Far below weight size; no official path.
RTX PRO 6000 Blackwell	96 GB	One card: no official fit. Four-card profile: marked verified by vLLM with reduced concurrency.
A100	40 or 80 GB	No validated path. An open issue reports an official-FP8 load failure on 2x A100.
H100	80 or 94 GB	One card: no. Multi-GPU capacity may be sufficient, but the recipe does not publish a validated H100 layout.
H200	141 GB	One card: weight arithmetic alone is insufficient. Official recipe uses 8x H200 TEP8.
B200	180 GB decimal, 167.638 GiB	The 172.782 GiB FP8 files exceed one GPU before runtime overhead. PLE offload might change capacity arithmetic, but no single-B200 validation is published.
GB300	288 GB HBM3e	TP2 is the documented minimum; TP4/TEP4 recommended. TP1 is not recommended.
MI355X	288 GB HBM3e	Official recipe uses 4x MI355X. No single-GPU result is published.
Mac Studio M3 Ultra, 512 GB unified	Capacity on paper for BF16, but vLLM’s initial implementation does not support Apple Silicon and no MLX/Metal recipe is official.

“No” in this table means the official checkpoint and documented runtime path, not every future community quantization. The live vLLM recipe now includes a third-party NVFP4 variant and there are early community efforts for DGX Spark. Those are different artifacts with different code and evidence. Do not transfer their fit claims to Qwen’s official FP8 repository.

QSA speed claims: why two official pages disagree

The Qwen launch post and technical report say that at one million tokens QSA achieves 7.6x prefill and 4.9x decode attention-module speedups over dense GQA. The report provides the missing conditions: a 16K chunk at batch size 1 for prefill, and batch size 4 with three MTP prediction steps for decode. The comparison includes the QSA indexer and sparse core attention.

The current vLLM recipe instead says 10.2x prefill and 6.6x decode at one million tokens, without publishing the benchmark conditions beside that sentence. We found no matching values in Qwen’s 28-page technical report. Until vLLM adds a reproducible measurement or correction, use Qwen’s 7.6x/4.9x figures with their stated test conditions and treat the recipe’s larger pair as unresolved.

This discrepancy does not change the hardware guidance. It changes how confidently anyone should repeat a launch number.

Verify the server before benchmarking it

Start with a text request that exercises the reasoning parser:

from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")
response = client.chat.completions.create(
    model="Qwen/Qwen3.8-Flash-Next-FP8",
    messages=[{
        "role": "user",
        "content": "Explain how Gated DeltaNet and Qwen Sparse Attention complement each other.",
    }],
    max_tokens=1024,
)
print(response.choices[0].message.content)

Then run tool-calling and multimodal smoke tests separately. Record the exact prompt, media preprocessing settings, output, server log and peak memory. A successful startup does not prove prefix caching, MTP, 262K context, tool calls and vision all work together.

Troubleshooting the preview runtime

Symptom	First check
Out of memory while loading	Confirm the exact checkpoint; increase TP; enable supported PLE CPU offload; reduce reserved context only if cache allocation is the failing component.
TP1 compilation OOM on GB300	Use TP2 or TP4, as the recipe recommends.
Eight H200 GPUs fail with FP8	Confirm `--enable-expert-parallel` and `--moe-backend triton`; plain TP8 is documented as incompatible.
Mamba/cache capacity error at startup	Keep `--max-num-seqs 256` or lower concurrency.
OOM on large multimodal batches	Keep `--gpu-memory-utilization 0.90` and reduce batch/concurrency.
MTP adds memory pressure	Reduce `num_speculative_tokens` below 3, then remeasure acceptance and throughput.
Pipeline-parallel startup failure	Use single-node TP or TEP; the initial N-gram implementation does not support pipeline parallelism.
DEP startup failure	Set `VLLM_PLE_CPU_OFFLOAD=1`; the recipe requires it for DEP.
XPU or TPU startup failure	These platforms are not supported by the initial implementation.
A100 FP8 dtype error	Treat A100 as unvalidated; follow upstream issue `#54219` rather than improvising production flags.
Prefix-cache illegal memory access	Reproduce against the pinned stock image and check current upstream Qwen3.8 issues before enabling production traffic.

FAQ

Is Qwen3.8-Flash-Next Qwen4?

No. It is an experimental preview of architecture Qwen intends to use for Qwen4. The released checkpoint is named Qwen3.8-Flash-Next.

How much VRAM does Qwen3.8-Flash-Next need?

The exact FP8 safetensors total 172.782 GiB, but the current vLLM recipe sets a 250 GB aggregate-VRAM minimum. BF16 tensors total 335.276 GiB, and the validated GB300 TP2 run used about 190 GiB per GPU.

Does 6B active parameters mean it has 6B-model memory requirements?

No. Active parameters describe per-token computation. The server must still store or offload the full expert weights, 51B N-gram table and other packaged tensors.

Can it run on one H100, H200, B200 or GB300?

No single-GPU configuration is officially validated. A single H100 is far too small. H200 and B200 require assumptions about PLE offload to get past weight arithmetic, and those TP1 paths are not documented. vLLM calls TP2 the minimum on GB300.

Can it run on an RTX 5090 or Apple Silicon?

Not with the official checkpoints and current vLLM recipe. The RTX 5090 has 32 GB. Apple offers enough unified memory in some M3 Ultra systems, but the initial vLLM implementation does not support Apple Silicon and there is no official Metal/MLX path for this checkpoint.

Can the 51B N-gram table be offloaded?

Yes, on supported NVIDIA builds. The vLLM recipe documents VLLM_PLE_CPU_OFFLOAD=1 and at least 51 GB of host memory plus runtime headroom. TP1 remains unvalidated and has an open community deadlock report.

Is FP8 quality materially worse?

Qwen says the official FP8 model is nearly identical in its performance metrics. Kingy did not independently reproduce that comparison, so the claim remains vendor-reported.

Does ordinary `pip install vllm` support it?

The model page shows a generic pip snippet, but the dedicated vLLM recipe says PyPI is not supported and its implementation PR was still open at this cutoff. Use the pinned preview image.

Can it use the full 262K or one-million-token context?

The model advertises 262,144 native tokens and a one-million-token YaRN extension. The vLLM recipe validated startup and bounded work with 262K configured, not a single full-length 262K request. Treat 1M as an explicit extension that needs workload testing.

Why does H200 require TEP8 instead of plain TP8?

The official FP8 checkpoint uses 128-wide quantization blocks. vLLM says plain TP8 is incompatible with that block layout. TEP8 adds expert parallelism while retaining eight tensor-parallel ranks.

Sources, method and limitations

Primary sources include Qwen’s August 26 launch post and technical report; pinned official BF16 and FP8 repositories; the vLLM recipe at commit 7997f1d; vLLM Docker documentation; implementation pull request #53896; and official NVIDIA, AMD and Apple hardware specifications.

Internal reading:

This edition did not rent accelerator time. It reports no Kingy tokens-per-second, time-to-first-token, peak VRAM or full-context result. Community issue reports are used only as warnings about unvalidated paths. We will update this guide when reproducible accelerator measurements are available.

Trending on Kingy

Qwen3.8-27B Runs on a Laptop—But Here’s the Hardware You Actually Need

Which Qwen3.8-27B GGUF Should You Download? Q2–Q8 Quantization Guide

Can MiniMax H3 Generate Uncensored Video? What Local Deployment Actually Changes