Trending on Kingy
Keep reading with the stories getting the most attention now.
As of August 1, 2026, DeepSeek V4 Flash on DGX Spark is possible on one box only as a heavily quantized derivative; the official 166.887 GB checkpoint does not fit. Two Sparks can hold the official weights, though serving still depends on patched software. Full Kimi K3 is not credible on one, two, or four Sparks: its 1.561 TB checkpoint crosses the raw capacity threshold only at roughly 12–13 boxes, depending on memory accounting, and about 16 is the first sensible planning point. Calling either API from a Spark is remote inference, not local execution.
Evidence cutoff: August 1, 2026, America/Vancouver. This is source and engineering analysis, not a hands-on Kingy.ai benchmark. No Kingy.ai hardware logs were produced for this article.
Evidence is identified where it matters: verified fact means a current owner or platform source; vendor claim means a capability or benchmark published by the vendor; community result means a documented third-party or runtime-maintainer run; and our calculation means arithmetic derived from published bytes, bandwidth, power or pricing.
The feasibility table
“Can it run?” can mean six different things. Conflating them is how a successful download turns into a claim of production readiness.
| Test | 1 Spark: DeepSeek V4 Flash | 2 Sparks: DeepSeek V4 Flash | 1–4 Sparks: Kimi K3 |
|---|---|---|---|
| Call the hosted API | Yes, but the inference is remote | Yes, but the inference is remote | Yes, but the inference is remote |
| Store official checkpoint on SSD | Yes; 166.887 GB fits the 4 TB SSD | Yes | Yes; 1.561 TB fits one 4 TB SSD |
| Hold official weights in memory | No | Yes, in aggregate | No |
| Run a lower-bit derivative | Yes; Q2-class, meaning roughly two-bit expert quantization, with fidelity and runtime caveats | Yes, with more headroom | No validated Spark artifact exists |
| Interactive single-user inference | Plausible; preview Q2 measured roughly 8–15 decode tok/s | Credible experimentally; current 0731 community results reach tens of tok/s | No credible evidence |
| Production-ready service | No | No current evidence supports that description | No |
The distinction between SSD capacity and memory residency matters. DGX Spark has enough storage for either download. Its SSD is not a second pool of GPU memory, and an engine that repeatedly fetches experts from storage is not equivalent to a resident model.
What a DGX Spark actually contains
DGX Spark is a compact Grace Blackwell system built around NVIDIA’s GB10. According to the official hardware guide, it combines an integrated Blackwell GPU with a 20-core Arm CPU and 128 GB of coherent LPDDR5X shared by the CPU and GPU. The published memory bandwidth is 273 GB/s.
That is one shared pool, not 128 GB of video memory plus separate system RAM. The operating system, display reservation, page cache, model weights, CUDA workspaces, graph allocations, activations and key-value (KV) cache all compete for it.
NVIDIA labels the memory “128 GB” but does not define the exact byte convention on the specification page. If the label is interpreted as decimal SI capacity, the conversion is:
128,000,000,000 bytes / 2^30 = 119.209 GiBThat is a unit conversion, not a measured physical-memory claim. Current community Linux reports expose roughly 121–122 GiB to the operating system after carve-outs, which is consistent with binary-sized DRAM rather than exactly 128 billion physical bytes. NVIDIA’s current DGX Spark release notes also specify a default 2 GB display reservation and dynamic unified-memory accounting. Capacity planning should use the actual machine’s MemTotal and runtime allocation logs; neither 128 GB nor 128 GiB is fully usable by model weights.
NVIDIA advertises up to one petaflop of four-bit floating-point (FP4) compute for Spark. The product page qualifies that figure as theoretical FP4 performance using sparsity. It is not a dense-compute number, sustained inference throughput or evidence that a given model has optimized kernels for this GPU.
Software compatibility is unusually important here. The host is Arm64 and the GPU target is SM121, as NVIDIA’s Spark porting documentation explains. A package can support “Blackwell” on x86 data-center GPUs without shipping an Arm64 wheel or SM121 kernel that works on Spark. Current successful DeepSeek deployments therefore pin exact branches, containers, compiler versions and kernel dependencies rather than relying on a generic installation command.
The current Founder’s Edition release notes list DGX OS 7.5.0, driver 580.159.03 and CUDA Toolkit 13.0.2; NVIDIA’s June update added three-Spark ring support in NVIDIA Collective Communications Library (NCCL) 2.30u1. Partner systems and generic DGX OS images can carry different versions, so benchmark reports need the actual host and container packages rather than a blended “current Spark stack.”
Spark’s 4 TB self-encrypting NVMe SSD is useful. It can hold both checkpoints separately, though Kimi leaves far less room for duplicate downloads, conversion workspaces and Hugging Face caches. The machine also has two ConnectX-7 ports rated at up to 200 Gb/s each. Those ports matter when two Sparks divide a model, but they do not turn the system into an NVLink supernode.
Total parameters and active parameters answer different questions
Both models are mixtures of experts (MoE). The router selects a subset of expert weights for each token instead of executing every expert. This reduces per-token work, but it does not make the rest of the checkpoint disappear.
DeepSeek describes the current V4 Flash 0731 checkpoint as a 284-billion-parameter model with 13 billion parameters active per target token. Its configuration has 43 layers, 256 routed experts, six selected routed experts per token and one shared expert.
Kimi K3 is much larger: 2.8 trillion total parameters and 104 billion active per token. It has 93 layers, 896 routed experts, 16 selected per token and two shared experts.
The active count is mainly a compute and weight-traffic fact. The total checkpoint is the storage and residency fact. A router may select different experts for the next token, the next prompt or the next user, so all routable experts must remain available somewhere: memory, another node or, in a much slower design, storage.
This is why “Kimi only activates 104B” does not make it a 104B model for memory planning. It is also why DeepSeek’s 13B-active figure does not imply that a 26 GB FP16 allocation can run the full model.
DeepSeek V4 Flash on DGX Spark: one box requires aggressive compression
The current checkpoint is DeepSeek-V4-Flash-0731, released on July 31. DeepSeek says it retains the preview’s target architecture and size but has been re-post-trained. It also includes the current DSpark speculative-decoding tensors. The official 0731 file tree contains 48 safetensor shards totaling:
166,886,535,336 bytes
= 166.887 GB
= 155.425 GiBThe indexed tensor payload is 166,878,536,440 bytes, or 166.879 GB; the roughly 8.0 MB difference is shard-container overhead. The on-disk sum answers download and SSD questions, while the tensor payload is the cleaner first-order memory input.
The checkpoint is 155.425 GiB. Even the generous comparison against a full 128 GiB pool, before display and operating overhead, leaves this deficit:
155.425 GiB - 128 GiB = 27.425 GiB
155.425 GiB / 128 GiB = 1.214×The gap is larger against the roughly 121–122 GiB exposed by current systems. No cache setting, unified-memory flag or model-parallel option makes that deficit disappear. An offloaded process might eventually execute, but the official weights are not resident on one machine.
Bandwidth provides a separate ceiling from capacity. If all 13 billion active parameters occupied only four bits, an optimistic lower bound would be:
13 billion × 0.5 byte = 6.5 GB of active weights per target-model step
273 GB/s / 6.5 GB = 42 target-model steps/sThat is our calculation, not a benchmark. It assumes perfect sequential use of published bandwidth and ignores higher-precision tensors, attention, routing, cache traffic, dequantization and kernel overhead. Speculative decoding can verify several proposed tokens per target step, so target-model steps and delivered output tokens are not interchangeable. Two nodes also do not simply double this ceiling: the weights and memory bandwidth are partitioned, while tensor-parallel collectives add network work.
What can fit
DeepSeek’s official checkpoint is already mixed precision: MoE experts use FP4 while most other parameters use FP8. It is not a uniform four-bit file. Scales, packed representations, higher-precision tensors and the speculative module explain why it is larger than the naive 142 GB obtained from 284B × 4 bits.
| Representation | Approximate size | Fits one Spark? | Evidence boundary |
|---|---|---|---|
| Official 0731 mixed checkpoint | 166.887 GB | No | Verified shard sum |
| Current community Q4/MXFP4 GGUFs | Roughly 155–156 GB | No | File availability, not runtime validation |
| Preview hybrid Q2/Q4 | About 97.6 GB | Yes, tightly | Preview artifact only |
| Preview model-specific Q2 | About 86.7 GB | Yes | Preview artifact with measured Spark runs |
An ideal uniform three-bit version would be 106.5 GB before overhead. That number is not a recommendation: practical formats retain sensitive tensors at higher precision and require scales and alignment. A generic Q3 could consume most of the available pool before the runtime allocates a useful context. The demonstrated one-Spark path is a model-specific Q2 or asymmetric Q2/Q4 design, not an arbitrary “make it smaller” conversion.
The current file-size range comes from community conversions including Bartowski’s approximately 156.4 GB MXFP4 GGUF and Unsloth’s approximately 155.1 GB UD-Q4_K_XL build. Their presence proves that files exist; it does not prove equivalent quality or complete Spark runtime support.
For broader format and workstation comparisons, Kingy’s DeepSeek V4 Flash 0731 local-hardware guide covers the current file landscape. The Spark-specific conclusion is simpler: release precision and current Q4 files do not fit; aggressive Q2-class compression can.
What one-Spark performance has actually been measured
The detailed one-Spark evidence comes from an independent Classmethod run using DwarfStar. This is a community measurement from the preview lineage, not a 0731 benchmark. The exact upstream checkpoint snapshot was not reported. The test used DwarfStar commit a97e7a3, a roughly 81 GiB model-specific Q2-imatrix GGUF named DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf, one GB10 Spark, compressed KV and single-stream greedy generation.
| Revision and quantization | Hardware and engine | Context | Concurrency | Reported prefill | Decode | Memory note | Source |
|---|---|---|---|---|---|---|---|
| Preview lineage; exact snapshot unreported; model-specific Q2-imatrix | 1× DGX Spark; DwarfStar a97e7a3 |
2,048 | 1 | 64.9 tok/s, incremental | 13.2 tok/s | 52 MB compressed KV | Classmethod community run |
| Same | Same | 32,768 | 1 | 162.6 tok/s, incremental | 11.8 tok/s | 475 MB KV | Same source |
| Same | Same | 65,536 | 1 | 247.1 tok/s, one-shot | 11.4 tok/s | 926 MB KV | Same source |
| Same | Same | 131,072 | 1 | 165.2 tok/s, incremental +64K | 9.95 tok/s | 1.83 GB KV | Same source |
| Same | Same | 262,144 | 1 | 109.3 tok/s, incremental +128K | 7.94 tok/s | 3.63 GB KV; about 115 GiB peak system memory | Same source |
The prefill figures are not a clean scaling series because the test modes differ. A one-shot prefill and an incremental addition to an existing context should not be compared as if they were the same workload. The decode trend is clearer: generation fell from 13.2 tok/s at 2K to 7.94 tok/s at 262K, while the machine approached its memory ceiling.
Eight to 14 generated tokens per second is usable for private research, batch analysis and patient interactive work. It is not fast chat by current hosted-model standards. More importantly, those measurements do not establish that a new 0731 Q2 build preserves the official model’s coding reliability, routing behavior, tool-call syntax or long-context retrieval.
Quantization loss is unlikely to be uniform. Casual prompts can look healthy while rare expert knowledge, long reasoning traces or exact structured output deteriorate. A responsible 0731 Q2 evaluation would compare the same tasks against the official API or a higher-precision local reference, including code tests, schema validity, tool loops and retrieval at multiple context positions. That evidence does not yet exist at the breadth required to call one-Spark 0731 full-fidelity.
Two Sparks: enough memory, unfinished serving software
Using a conservative 128.000 GB planning budget per node, two Sparks provide 256.000 GB in aggregate. Subtracting the exact 0731 tensor payload gives:
256.000 GB - 166.87853644 GB = 89.12146356 GB
≈ 89.121 GB aggregate planning headroomThis is conservative planning arithmetic, not a claim that two Sparks behave like one 256 GB GPU. The headroom is split across two machines and must cover two operating systems, two serving processes, CUDA workspaces, network buffers, KV cache and speculative-decoding state. Working deployments prove that the partition can be made viable.
The usual design applies tensor parallelism across the two machines. Each node holds a partition of the model and participates in collectives as the token moves through the network. Spark’s clustering guide exposes two 200 Gb/s Ethernet ConnectX-7 ports and supports remote direct memory access over Converged Ethernet (RoCE) interfaces. One port’s raw line rate is 25 GB/s before protocol and collective overhead, far below the local 273 GB/s memory bandwidth.
There is a critical limitation: NVIDIA’s CUDA porting guide says GPUDirect remote direct memory access (RDMA) is not supported on DGX Spark. ConnectX-7 plus RoCE does not mean the network interface can directly and coherently access ordinary CUDA allocations. Working software may stage transfers through CPU-visible memory. That distinction matters for latency and CPU overhead, especially as node count and layer count grow.
Stock vLLM versus working forks
As of the evidence cutoff, upstream vLLM’s SM120/SM121 DeepSeek V4 fixes remained in open pull request #41834. Functional Spark deployments use that maintained branch or downstream patches and containers. The stable two-Spark 0731 validation used Torch 2.13.0, Triton 3.7.1, FlashInfer 0.6.15.post1 and NCCL 2.30.7. The live branch then moved to FlashInfer 0.6.16; its maintainer warns that installing it can restore NCCL 2.29.7 unless 2.30.7 is explicitly re-pinned on every node. These are moving branch facts, not universal requirements.
The packaging problem is broader than one PR. vLLM issue #36821 tracks Arm64 and SM121 kernel gaps, while issue #45317 covers sparse-attention backend selection on SM121. Some generic SM12x support has improved, so the oldest “nothing works” wording is stale. The accurate description is: working forks exist, but the stock path is unfinished.
SGLang supports DeepSeek V4 on several data-center GPUs, but its current DeepSeek V4 cookbook does not validate GB10 or DGX Spark. Upstream llama.cpp recognizes the DeepSeek 4 architecture, yet the most mature Spark evidence still comes from specialized GGUFs and model-specific projects such as DwarfStar/ds4. DwarfStar calls itself beta and is not a generic production GGUF server.
Current two-Spark performance
The most relevant current result is a community 0731 deployment using patched downstream vLLM. It used the native mixed weights, NVFP4 KV cache and built-in DSpark speculation across two Sparks with tensor parallelism.
The report’s workload-specific decode results were:
| Revision and quantization | Engine and hardware | Context | Concurrency | Workload | Prefill | Reported decode | Source |
|---|---|---|---|---|---|---|---|
| 0731; official mixed weights; NVFP4 KV cache | Patched downstream vLLM; 2× DGX Spark; tensor parallel 2; DSpark block size 5 | 1M configured; actual prompt length not reported | 1 | Prose reasoning | Not reported | 37.8 tok/s | Two-Spark 0731 community report |
| Same | Same | 1M configured; actual prompt length not reported | 1 | Code generation | Not reported | 62.2 tok/s | Same source |
| Same | Same | 1M configured; actual prompt length not reported | 1 | Structured/repetitive | Not reported | 66.1 tok/s | Same source |
| Same | Same | 1M configured; short counting prompt | 1 | Repetitive peak finder | Not reported | 78.4 tok/s warm; 56.8 tok/s immediately after cold start | Same source |
| Same | Same | 1M configured; benchmark mix not fully reported | 1 | Patch 4 mean | Not reported | 55.4 tok/s | Same source |
These are community measurements from one patched system, not independent lab results. They nevertheless show that two Sparks can provide genuinely interactive single-user inference with the current checkpoint.
The variation is not noise. Speculative decoding proposes multiple tokens and asks the target model to verify them in a single step. Repetitive or predictable output produces high acceptance; open-ended prose produces less. On the same 0731, native-weight, NVFP4-KV, patched-vLLM, two-Spark configuration, the single-user table showed 60.2% mean acceptance. A separate 35-minute pool of agent traffic showed 56.1%; its prompt lengths and concurrency were not reported. A headline based only on the warm 78.4 tok/s repetitive case would misrepresent normal use.
There is also an unresolved configuration mismatch. DeepSeek’s 0731 model card suggests seven speculative tokens, while the repository configuration has a DSpark block size of five and current Spark branches favor five. The article should not pretend that one setting is universally correct until the owner documentation and engine implementation converge.
For baseline performance without importing 0731 speculation results, the maintained vLLM branch reports the following preview-checkpoint clean-build runs. These are runtime-maintainer community results, not independent findings:
| Revision and quantization | Engine and hardware | Context | Concurrency | Prefill | Decode | Source |
|---|---|---|---|---|---|---|
| Preview; official mixed weights; FP8 KV cache; MTP speculation with 2 draft tokens | vLLM PR #41834 branch; 2× DGX Spark; tensor parallel 2 | 8K | 1 | 1,650 tok/s | 38.4 tok/s | Open vLLM PR #41834 |
| Same | Same | 16K | 1 | 1,776 tok/s | 38.6 tok/s | Same source |
| Same | Same | 32K | 1 | 1,672 tok/s | 38.5 tok/s | Same source |
They help bound this particular hardware and branch, but they are not substitutes for controlled 0731 testing. They also cannot be merged with the DSpark-assisted 0731 rows above: the revision, speculative method and KV format differ.
One million tokens: supported is not comfortable
DeepSeek’s architecture advertises a 1,048,576-token context. A local engine accepting --max-model-len 1048576 proves only that the configuration parsed.
The patched two-Spark work configured a 1M per-request ceiling and, in a preview run, exposed an approximately 1.9M-token shared KV pool. That does not mean multiple simultaneous one-million-token conversations fit. Aggregate server throughput is not single-user decode speed, and a shared 1.9M-token cache cannot contain even two full 1M sequences. An under-specified six-session aggregate result was excluded from the benchmark table because it did not report enough row-level context and workload detail.
A credible 1M claim needs the actual input size, cold prefill time, time to first token, peak memory, KV precision, retrieval accuracy by document position, final-context decode speed and repeated successful completions. No current evidence establishes that 1M on two Sparks is a comfortable general-purpose serving configuration. It remains technically plausible, expensive to prefill and sensitive to cache and workload choices.
DSpark is not DGX Spark
DSpark is DeepSeek’s confidence-scheduled speculative-decoding module. DGX Spark is NVIDIA’s GB10 computer. The shared word “Spark” is a naming collision. DSpark is not an NVIDIA runtime, and its presence in the checkpoint does not imply special DGX Spark support.
Kimi K3 is a rack-scale model, not a larger DeepSeek experiment
Kimi K3 is open-weight under the custom Kimi K3 License. It is not MIT- or Apache-licensed. Moonshot’s official repository describes a 2.8-trillion-total, 104-billion-active multimodal MoE with a 1,048,576-token context. The 93 transformer layers split their attention blocks into 69 Kimi Delta Attention layers and 24 gated multi-head latent attention layers. The first feed-forward layer is dense; the sparse blocks expose 896 routed experts, select 16 per token and add two shared experts.
The official Kimi K3 file tree contains 96 safetensor shards totaling:
1,560,936,091,448 bytes
= 1.560936 TB
= 1,453.735 GiBThe tensor payload, excluding shard headers, is 1,560.860 GB. That is the useful first-order residency number. The table below deliberately uses a conservative 128.000 GB planning budget per Spark; it does not assert that this is the exact physical byte count or the memory a process will receive.
| Spark count | Aggregate planning memory | Headroom after K3 tensor payload | Assessment |
|---|---|---|---|
| 1 | 128 GB | -1,432.860 GB | Not close |
| 2 | 256 GB | -1,304.860 GB | Not close |
| 4 | 512 GB | -1,048.860 GB | Still less than one-third of requirement |
| 12 | 1.536 TB | -24.860 GB | Fails under the conservative planning basis |
| 13 | 1.664 TB | 103.140 GB total; 7.934 GB/node | Conservative raw mathematical minimum |
| 16 | 2.048 TB | 487.140 GB total; 30.446 GB/node | Sensible capacity-planning point only |
| 18 | 2.304 TB | 743.140 GB total; 41.286 GB/node | More headroom, still no validated topology |
Thirteen comes from ceil(1,560.860 / 128). A favorable calculation using roughly 121.7 GiB exposed per node, equivalent to about 130.7 decimal GB, lets 12 nodes barely cross the tensor-payload line, with only about 7 GB left across the cluster. That observed-memory basis varies by firmware, display reservation and runtime, so the honest raw threshold is 12–13 Sparks, not a universal 13. Neither result is a deployment recipe. Under the conservative 13-node basis, less than 8 GB per node remains for the operating system, runtime, communication buffers, workspaces, activations and KV cache. Sixteen is a more defensible memory-planning number, not evidence that 16 Sparks can serve K3 correctly or quickly.
NVIDIA’s guided Sync topology covers up to three directly connected Sparks or four through a switch. A larger manual fabric is conceivable, but a 13–18-node design moves far outside the guided configuration. It also asks 93 model layers to communicate across Ethernet without GPUDirect RDMA. Equal aggregate bytes do not make 18 Sparks equivalent to eight high-bandwidth data-center accelerators.
“Quantize it to four bit” is not an answer
Moonshot says K3 uses MXFP4 weights and MXFP8 activations after quantization-aware training. The official configuration excludes attention, shared experts, the dense MLP, LM head, vision tower and multimodal projector from the packed four-bit rule. That is why the artifact is 1.561 TB rather than the ideal 1.400 TB obtained by multiplying 2.8T parameters by half a byte.
K3 is already aggressively compressed where most of its routed expert volume lives. “Make it Q4” does not reveal another easy 4× reduction.
A more aggressive conversion is mathematically conceivable. The dossier’s tensor-informed estimates—not released artifacts—produce roughly these capacities:
| Hypothetical K3 format | Estimated artifact | Minimum at 128 GB/node | Count at 102.4 GB/node, or 80% of that budget |
|---|---|---|---|
| Native mixed MXFP4 | 1.561 TB | 13 | 16 |
| Routed experts compressed toward 3-bit | About 1.221 TB | 10 | 12 |
| Routed experts compressed toward 2-bit | About 0.880 TB | 7 | 9 |
These are calculations, not measurements. No owner-supplied or rigorously validated three- or two-bit K3 Spark checkpoint was available at the cutoff. The table says nothing about routing stability, accuracy, conversion memory, kernels or whether the resulting model can complete a request.
There is also no official smaller or distilled K3. Moonshot’s separate Kimi-Linear-48B-A3B-Instruct is an earlier research model, not a K3 distillation. Kingy’s Kimi K3 hardware and VRAM analysis covers the broader workstation and data-center ladder.
SSD expert streaming does not make K3 interactive
A hybrid MoE engine can keep dense and frequently used tensors resident while fetching cold experts from SSD. That can turn an impossible allocation into a proof of life. It does not eliminate storage latency or random expert access.
Take an optimistic 7 GB/s sequential SSD assumption. If K3’s 104B active parameters required only four bits each, the lower-bound active-weight traffic would be:
104 billion × 0.5 byte = 52 GB per target token
52 GB / 7 GB/s = 7.43 seconds per token
= 0.135 token/sThis already ignores higher-precision tensors, attention, KV traffic, dequantization, routing and random I/O. Real expert access is not one perfect sequential read. A full sequential scan of the 1.561 TB artifact at the same optimistic rate would take about 223 seconds.
Caching hot experts can improve a particular workload, and speculation might amortize target steps if compatible support existed. Neither changes the conclusion: an engine that eventually emits one Kimi token after seconds of storage traffic has demonstrated technical persistence, not useful local inference.
Official guidance points elsewhere
Moonshot recommends supernodes with 64 or more accelerators to keep communication inside a large high-bandwidth domain. That is vendor guidance for production-class infrastructure, not a minimum byte count.
The vLLM K3 launch and recipe requires vLLM 0.27 or newer, CUDA 13, an R580-or-newer driver and K3-specific dependencies. Its smallest listed shapes use eight B300 or GB300 accelerators, or eight AMD MI350X/MI355X accelerators; 16-accelerator B200 and GB200 configurations are also documented. The SGLang K3 cookbook lists B300/GB300, B200/GB200, H100/H200 and MI35X shapes, but warns that no preset had completed a full serving round on the final K3 weights at the evidence cutoff. Neither project provides a DGX Spark recipe.
No credible Kimi K3 benchmark on DGX Spark was found. The vLLM launch post does provide reproducible single-user TP8 data on eight GB300 NVL72 accelerators, which establishes the performance class of a supported data-center system:
| Revision and quantization | Engine and hardware | Context and workload | Concurrency | Prefill | Decode | Run state | Source |
|---|---|---|---|---|---|---|---|
| Final public Kimi K3; native MXFP4 | vLLM K3 launch container/recipe; 8× GB300 NVL72; TP8; exact runtime commit not disclosed | Random prompts; 8,192 input and 1,024 output tokens; no speculation | 1 | Not reported | 111 tok/s | Warm/cold state not reported | vLLM K3 vendor benchmark and reproduction command |
| Same; Inferact Kimi K3 DSpark draft | Same; TP8; 7 speculative tokens | SPEED Bench throughput_16k; input capped at 10,240; 1,536 output tokens; low-entropy category |
1 | Not reported | 331 tok/s | Warm/cold state not reported | Same source |
These are vendor serving-stack results, not independent evidence. The speculative and non-speculative rows use different datasets, so 331 / 111 is not a controlled same-prompt speedup calculation. More importantly, eight GB300 NVL72 accelerators have a radically different memory and communication system from GB10; neither number predicts a Spark cluster.
The tradeoffs that the fit/no-fit answer misses
Fidelity
One-Spark DeepSeek pays for capacity with aggressive quantization. Until a current 0731 Q2 build is evaluated against the official API across code, tool use, structured output and long context, fidelity is an open risk. Two Sparks can retain the owner checkpoint and avoid that particular compromise. Kimi on Sparks requires either an enormous native cluster or an unvalidated second round of quantization.
Latency, prefill and decode
Prefill processes the prompt; decode generates new tokens. A server can prefill quickly and decode slowly, or restore a cached prefix quickly after a terrible cold first run. Report both phases separately.
The one-Spark preview evidence shows decode worsening as context grows. The two-Spark 0731 evidence shows good interactive decode with DSpark, but acceptance depends on content. Neither provides a controlled, independent 0731 study from short prompts through a full 1M context.
Context and concurrency
A million-token architecture is not a promise that a million-token request is cheap, fast or accurate. KV-cache precision, prefill duration, memory fragmentation and retrieval quality all matter. Concurrency divides the cache and introduces scheduling pressure. Six ordinary sessions sharing a 1.9M-token pool is not six million-token sessions.
Privacy and offline operation
Local inference removes one external model provider from the data path and can operate without an internet connection. Those are real advantages. Local does not automatically mean private: prompts can remain in logs, disk-backed KV caches, shell history, crash dumps, monitoring and swap. Third-party quantizations, containers and moving forks are supply-chain dependencies.
Reliability and maintenance
The Spark software path is a research stack. Operators must pin model revisions, runtime commits, CUDA and NCCL versions, container digests and quantization hashes. A branch update can change kernels or restore a different NCCL build. A two-node system also adds link configuration, synchronized dependencies and node-failure behavior.
A process serving several requests is not production validation. The missing evidence includes long soak tests, request cancellation, mixed long/short traffic, OOM recovery, network interruption, rolling upgrades, authentication, observability and failover.
Power
NVIDIA specifies a 140 W GB10 system-on-chip (SoC) thermal design power and a 240 W external supply. Its regulatory measurements list 233.2 W maximum and 38 W idle. The maximum is not a measured LLM inference draw.
For a boundary calculation at $0.15/kWh:
0.2332 kW × 8,760 hours × $0.15/kWh
= $306.42 per Spark-yearActual cost depends on utilization and wall power. A 16-Spark Kimi thought experiment also adds networking, cooling and idle consumption across the cluster; electricity is still likely smaller than hardware and operator cost.
Licensing
DeepSeek publishes the 0731 repository under the MIT License.
Kimi K3 is open-weight under the custom Kimi K3 License. A Model-as-a-Service operator whose MaaS business revenue, together with its affiliates, exceeds $20 million during any consecutive 12-month period must obtain a separate commercial agreement. A commercial product above 100 million monthly active users or $20 million monthly revenue must prominently display “Kimi K3,” subject to the license’s definitions and exceptions for internal use and the official Kimi service. Kingy’s Kimi K3 license analysis explains those conditions in more detail. This is not legal advice.
Hardware and API economics
The current US Founder’s Edition price is $4,699 from NVIDIA. Two cost $9,398 before tax, cables, switches, support or engineering time. The same list-price arithmetic gives $56,388 for 12, $61,087 for 13, $75,184 for 16 and $84,582 for 18 Sparks. Those Kimi totals exclude the network and operational platform required to make the machines behave as a cluster.
DeepSeek’s OpenAI- and Anthropic-compatible API exposes the public-beta deepseek-v4-flash model with a one-million-token context and up to 384K output. Its current API pricing is $0.0028 per million cache-hit input tokens, $0.14 per million uncached input tokens and $0.28 per million output tokens. Moonshot exposes kimi-k3 through compatible endpoints with a one-million-token context; current K3 pricing is $0.30 cached input, $3 uncached input and $15 output per million tokens.
For an illustrative workload with five uncached input tokens per output token:
DeepSeek per 1M output-equivalent tokens
= 5 × $0.14 + 1 × $0.28
= $0.98
Kimi per 1M output-equivalent tokens
= 5 × $3.00 + 1 × $15.00
= $30.00These are calculations using August 1 prices, not break-even recommendations. Cache hits, reasoning tokens, retries and future price changes can move the result. Local and hosted outputs may also differ in precision, runtime and quality.
DeepSeek’s API is especially difficult to beat on token cost alone. One $4,699 Spark equals about 4.8 billion DeepSeek output-equivalent tokens at the illustrative 5:1 ratio before electricity, labor, downtime or quality differences. Local DeepSeek is better justified by privacy, offline availability, model control, experimentation or sustained capacity needs than by a simplistic “tokens become free” claim.
Kimi’s API is far more expensive per token, but a credible native deployment also begins with data-center hardware. A hypothetical 16-Spark cluster is not automatically the cheaper alternative because no evidence shows that it can provide comparable K3 service.
What each reader should do
If you already own one Spark: Use it as an experimental DeepSeek machine. Choose a documented model-specific Q2 or hybrid build, start around 32K context, record exact artifact and runtime revisions, and compare important outputs with the official API. Do not expose it as a production endpoint or assume preview benchmark speed and quality carry over to 0731.
If you are considering two Sparks: Buy the second unit only if native-checkpoint DeepSeek research, CUDA/Linux alignment or distributed-serving work is itself valuable. Two machines can hold the official weights and deliver interactive performance, but today that means maintaining a patched stack. It is not a plug-and-play appliance or an obvious financial substitute for the API.
If you specifically want Kimi K3: Use Moonshot’s API or rent a documented B300/GB300, MI350X/MI355X, B200, GB200 or comparable deployment. Do not buy a 12–18-Spark cluster because one memory division says the weights fit. That arithmetic ignores the serving fabric, kernels, 93 layers of communication and the complete absence of a Spark benchmark.
If your only goal is cheap inference: Start with DeepSeek’s API. Measure one month of real cached input, uncached input, output, latency and availability. Buy local hardware only if that workload, or a concrete privacy, offline-use or control requirement, justifies it.