Can a 27B AI Model Really Run on an iPhone? Bonsai’s 3.9GB Reality

Evidence audit · Source snapshot: July 31, 2026 · Vendor and independent results clearly separated

One-sentence verdict

Yes—the one-bit weights genuinely fit and independent developers have run Bonsai 27B on 8GB iPhones at roughly 8–9 tokens per second, but PrismML’s exact 11 tok/s Pro Max result remains vendor-only, its official iOS path is still developer-grade, and the one-bit model gives up too much instruction, tool and vision reliability to make “27B on a phone” synonymous with full 27B-class usefulness.

The four-part verdict

This is no longer a vaporware story. A 27B-derived model really can generate interactively on a phone. But “the weights load” is only the first gate. Memory headroom, public buildability, sustained speed and retained capability determine whether the result is an engineering milestone or a useful product.

What Kingy inspected

For this investigation, Kingy inspected the live GGUF and MLX repositories, exact file trees, model configuration, PrismML’s July 14 whitepaper, the Bonsai demo repository, PrismML’s llama.cpp, MLX and MLX Swift forks, relevant upstream pull requests, commit histories, demo captions, Apple memory documentation, third-party apps, and independent source code and raw iPhone logs. The source snapshot is dated July 31, 2026.

Kingy did not run Bonsai on an iPhone and does not present any PrismML, RunAnywhere or community number as a Kingy measurement. A signed-in ChatLLM catalog check did not expose Bonsai, so ChatLLM could not supply an independent run and no ChatLLM output is used as evidence.

What “27B on an iPhone” actually means

Bonsai 27B is not a 3.9GB file merely named after a bigger model. Its language architecture is derived from Qwen3.6-27B and retains about 27.3 billion language parameters: roughly 24.8B in 64 blocks plus 2.5B in the embedding and output head. A separate 0.46B vision tower brings the multimodal total to about 27.8B parameters. PrismML represents the binary language weights as +1 or −1, with one FP16 scale shared by each group of 128 weights. That works out to an ideal 1.125 bits per weight, not literally one bit once scale metadata is counted. PrismML’s model card says the low-bit treatment covers embeddings, attention projections, MLP projections and the LM head.

That distinction matters: this is a real extreme-compression achievement, not a small model in a 27B trench coat. It is also not equivalent to storing the original FP16 values more compactly. The weights have been transformed into a much narrower representation, and the quality trade-offs are measurable.

The famous 3.9GB is a rounded description of the binary language weights. The live GGUF file is 3.803GB on disk and contains about 3.792GB of resident tensor payload. The public MLX model.safetensors file—the format pointed at for Apple devices—is 5.129GB, because it also bundles vision tensors and carries MLX scale-and-bias packing overhead. The full MLX repository is about 5.159GB. “3.9GB on an iPhone” therefore does not describe the actual download in every runtime, the app binary, the tokenizer, the KV cache, activations, Metal workspaces, the operating system, a vision pass, or speculative decoding.

The phone claim is also narrower than PrismML’s demos can make it look. The whitepaper’s 11 tok/s row is text-only, batch one, depth zero, with no drafter and no vision tower. The launch video labels its multimodal phone segment “Cached & Prefilled Image Context”—evidence of post-prefill generation, not cold image encoding, first-token latency or peak multimodal memory. PrismML’s launch page makes that qualifier visible in the demo caption.

Complete peak-memory budget

The 3.9GB headline stops at the easiest line item. A phone must hold the resident weights, runtime and activation buffers, and a KV cache that grows with context. Vision and speculative decoding are additional residents when enabled.

All sizes below are decimal GB unless marked GiB. File sizes come from the live Hugging Face trees at the pinned revisions; memory peaks are classified as vendor-measured or derived.

On-disk components

The public MLX file and GGUF packs are not interchangeable measures of the same download. PrismML’s whitepaper explains that MLX stores both a scale and a bias for each group and bundles the vision tower in the same safetensors file.

KV cache at 4K, 32K, 100K and 262K

Only 16 of the model’s 64 layers use full attention. With four KV heads, 256 dimensions per head, and both keys and values, that is 32,768 cached values per token. FP16 needs 64KiB per token. Ideal raw four-bit storage would need 16KiB, but the live formats carry block scales and, in MLX, affine metadata. llama.cpp Q4_0 uses 18KiB per token, while MLX documents 3.56× compression for affine four-bit—closer to 18KiB than 16KiB.

PrismML’s current demo notes now describe four-bit KV as experimental on the llama.cpp backend and explicitly say it reduces the cache by about 3.5×, from 64KiB to roughly 18KiB per token. That is more conservative than the whitepaper’s clean 4× arithmetic. See the pinned KV-cache notes.

Realistic language-only peak memory

These estimates combine the parsed resident tensor sizes, source-derived cache sizes and runtime overhead calibrated from PrismML’s table—about 1.14–1.25GB on GGUF and 1.42–1.44GB on MLX. They are not Kingy device measurements, and no inspected independent test published a complete iPhone peak-footprint trace.

Now the key correction: PrismML’s 5.2GB at 4K and 11.6GB at 100K are vendor measurements for the llama.cpp/GGUF path with uncompressed KV. The whitepaper’s 6.8GB at 100K and 9.4GB at 262K with four-bit KV are projections obtained by shrinking the cache term by an ideal 4×; they are not measured iPhone peaks. The live formats imply closer to 6.9GB and 9.9GB for GGUF, or roughly 7.5GB and 10.5GB for MLX in compressed steady state.

There is a second, larger problem at full context. The documented public MLX Swift LM cache path stores the prompt cache in raw FP16 during prefill, then compresses it on the first decode step. Unless PrismML’s unpublished iPhone harness progressively compresses the cache differently, a 262K prompt could transiently approach the ~22.8GB FP16-peak column before settling near 10.5GB. No public PrismML log demonstrates otherwise. That makes “262K on an iPhone” an architectural/configuration claim, not a reproduced phone workflow.

Optional-component peak increments

At 4K, a text-only GGUF run is around 5.2GB. A realistic image-capable peak is around 6.2GB; adding the drafter pushes the lower bound toward 7.0GB; enabling both approaches 8.0GB before uncounted component workspaces. At full context with actual Q4 cache, the GGUF lower bounds become roughly 9.9GB text-only, 10.9GB with vision, 11.8GB with the drafter, and 12.7GB with both.

Those totals explain why the weights can fit while optional features do not. In the independent base-iPhone test, the 3.53GiB target ran, but loading the 1.66GiB drafter caused iOS to terminate the app.

Setup and runtime reproducibility

The model files are public and Apache-2.0 licensed. PrismML’s software story is less complete.

At the July 31 snapshot:

• The Bonsai demo repository has practical Mac, Linux and Windows scripts, pinned Python/MLX guidance and desktop benchmark submissions. It has no complete Bonsai iOS app, Xcode project or raw iPhone benchmark log.
• PrismML’s MLX Swift fork contains the low-bit Swift runtime kernels, not a Bonsai phone application or benchmark harness.
• PrismML’s llama.cpp fork publishes an iOS XCFramework, but the official demo repo does not include an example app that consumes it.
• Mainline support is still moving. PrismML’s core MLX pull request remained open, while its MLX Swift pull request was closed pending propagation through core MLX.

The official iOS quickstart is essentially: use the MLX Swift fork. It does not specify an app target, model download integration, entitlement, benchmark command, exact Swift revision or expected raw output. A capable iOS developer can assemble the pieces; a normal user cannot reproduce PrismML’s exact test from PrismML’s instructions alone.

The independent build changes the verdict

On July 17, a Japanese developer published a substantially stronger reproduction: a hands-on report backed by an MIT-licensed repository at commit 48f4377. It contains a patched llama.swiftui app, build steps, an HTTP API, speculative-decoding fixes, evaluation outputs and raw benchmark JSON.

The test used a base iPhone 17 (iPhone18,3) with 8GB RAM, iOS 26.4.2, a 4K context, Q4_0 KV, full Metal offload and a provisioning profile carrying Apple’s increased-memory entitlement. Results included 9.0 tok/s over a 256-token chat generation, 7.82 tok/s on TG128 and 6.55 tok/s over TG512. That is not the same device or runtime as PrismML’s result, but it independently proves that the central physical claim is real.

It also exposes the rough edges. Cold PP2048 and warm PP1024 triggered a Metal GPU error/recovery on that runtime snapshot, corrupting context until restart. The drafter jetsammed the 8GB phone. The reproduction is credible precisely because it publishes the failures as well as the successful run.

Third-party apps lower the barrier further. RunAnywhere ships Bonsai support and an on-device benchmark screen; Atomic Chat also exposes the model through an App Store client. That moves Bonsai beyond a private developer demo. But a third-party app reproduces “the model runs,” not PrismML’s precise MLX Swift harness or its 11.0 tok/s figure.

iPhone 17 Pro or Pro Max?

The documentation contradicts itself. PrismML’s July 14 press release says iPhone 17 Pro and claims 11 tok/s. The whitepaper’s throughput table, battery table and methodology say iPhone 17 Pro Max (A19 Pro). The launch page’s multimodal demo caption also says Pro Max. No separate PrismML Pro benchmark, raw log or second hardware configuration is disclosed.

The defensible resolution is simple: PrismML’s documented benchmark device was the iPhone 17 Pro Max; “Pro” in the press copy is shorthand or an editing error. RunAnywhere’s later 10 tok/s Pro result is a separate test and does not repair PrismML’s documentation discrepancy.

Short-prompt versus long-context performance

PrismML’s 11.0 tok/s figure is a TG128 measurement: 128 generated tokens after a 512-token prompt, at depth zero and batch one, using MLX Swift on the Pro Max. Its PP512 result is 111 tok/s. Both exclude vision and the drafter. PrismML says runs were repeated and averaged and that phone decode used a palindrome ordering to separate cold and warm behavior, but it has not published the raw iPhone runs.

The independent base-iPhone figures bracket a more conservative experience: 9.0 tok/s for one chat run, 7.82 tok/s over TG128 and 6.55 tok/s over TG512. RunAnywhere reports 10 tok/s and a 1.813-second time to first token on an iPhone 17 Pro using CPU llama.cpp. RunAnywhere is a runtime vendor, not a neutral lab, but the result independently supports PrismML’s order of magnitude.

So the speed claim is credible, but “independently reproduced at 11 tok/s” would be false. No one we found repeated 11.0 tok/s on the same Pro Max, MLX Swift revision, model pack, prompt, thermal state and benchmark harness.

Long context is a different workload. At PrismML’s reported 111 tok/s PP512 rate, ingesting 100K tokens would take about 15 minutes and 262K about 39 minutes, even if throughput stayed flat. It probably would not. The independent base iPhone’s PP512 fell from 74.3 tok/s cold to 40.7 tok/s warm, and its longer prefills hit GPU recovery at only 1K–2K tokens on that fork snapshot.

PrismML’s four-bit KV quality experiment also stops short of the headline. It measures output divergence against FP16 KV on MATH-500 generations and BABILong at 16K; it does not demonstrate 100K or 262K retrieval accuracy, time to first token, sustained throughput, prefill peak memory or stability on an iPhone.

The right wording is: Bonsai supports a 262K architectural context and PrismML projects compressed steady-state memory for it; no public evidence shows an iPhone successfully prefilling, retaining and using a 262K prompt.

Thermal throttling, battery use and sustained operation

PrismML’s only disclosed phone endurance test generated 3,360 tokens in 5.2 minutes on an iPhone 17 Pro Max using easymlx. Decode averaged 10.82 tok/s, the battery indicator fell from 100% to 95%, and iOS thermal state moved from nominal to fair. PrismML calls the energy numbers approximate rather than hardware-metered.

The whitepaper extrapolates 672 tokens per percentage point to roughly 67,000 tokens on a full charge. That projection is mathematically consistent but is not a full-discharge test. At 10.82 tok/s, 67,000 tokens would require about 103 minutes of uninterrupted generation. The observed five-percent drop also linearly implies about 58% battery per hour. Battery percentage is coarse, and the paper does not disclose ambient temperature, case, screen brightness, battery health or background load.

Independent sustained evidence is harsher. On the base iPhone 17, TG512 declined from roughly 7.8 to 6.25 tok/s as thermal state moved to fair—a roughly 20% decode drop. Warm PP512 fell from 74.3 to 40.7 tok/s, about 45%. The base phone is not the Pro Max and does not share its vapor-chamber cooling, so these percentages should not be transferred directly. They do establish that a cold 128-token burst is not enough to characterize sustained on-device work.

No independent 20–60-minute Pro or Pro Max run, full battery discharge, surface-temperature trace or external power measurement was found. Bonsai looks viable for short sessions. The evidence is not yet strong enough for continuous agent loops, repository-scale coding or an always-on assistant without meaningful heat and battery cost.

One-bit versus ternary versus Q4 quality

“89.5% of FP16 intelligence retained” is the least useful way to read the quality data. It divides one unweighted average by another across 15 very different benchmarks. It does not mean every capability retains about 90%, and the tests are all PrismML measurements—not Kingy results.

The table below is Kingy’s category calculation from PrismML’s per-benchmark Table 14. FP16, Q4 and conventional Q2 were run at temperature 1.0; Bonsai used 0.7. Most tasks were single-pass, only AIME used eight samples, and some option orderings were unseeded. Those choices limit clean model-to-model inference.

The conventional Q4 build is essentially lossless in this vendor suite—84.99 versus 85.07—but it occupies 17.6GB before runtime overhead and cannot fit an iPhone. Ternary Bonsai is the more credible laptop compromise: it retains most math and coding performance and materially beats one-bit on instruction following, tools and vision, but its live GGUF is about 7.17GB and the vendor says it exceeds the roughly 6GB iOS app budget.

The one-bit model preserves math unusually well and retains much of coding. Its larger losses land exactly where an on-device “agent” needs reliability:

• Instruction following: −12.73 points versus FP16.
• Tool use: −13.97 points.
• Vision: −13.04 points.
• Knowledge/reasoning: −9.76 points.
• Coding: −6.86 points.
• Math: −3.67 points.

The worst individual vendor-reported drops are τ²-Bench (82.90 to 61.34), MMMU-Pro (79.94 to 60.48), IFBench (68.03 to 52.36) and LiveCodeBench (87.77 to 76.40).

The conventional Q2 comparison is even more revealing. One-bit beats IQ2_XXS overall because it is dramatically stronger in math and coding, but it is worse than Q2 in knowledge, instruction following, tool use and vision. The aggregate headline reverses depending on which capability matters.

And “one-bit vision” is shorthand, not a literal system description. PrismML keeps the separate vision tower at HQQ four-bit. The one-bit vision score combines that four-bit encoder with the binary language model.

Independent checks are smaller but directionally useful. A public 98-question Jetson benchmark scored Qwen3.6 Q4 at 94.2%, ternary at 86.2% and one-bit at 82.9%; it is a custom suite, not a replacement for standardized evaluation. The independent iPhone developer’s eight-prompt check found strong English logic, algebra and simple Python, but one Japanese recall task hallucinated, repeated itself and code-switched. Two easy prompts consumed the full 1,200-token allowance in thinking without returning an answer.

That makes the usefulness verdict narrower than the parameter count suggests: impressive short-form reasoning and coding for its footprint, but not a trustworthy local tool agent, vision analyst or multilingual factual assistant without task-specific validation and guardrails.

Privacy and offline-operation caveats

Once the model and runtime are downloaded, prompt evaluation and generation can happen entirely on-device and continue in airplane mode. That removes cloud-inference latency, marginal token fees and one major data path.

It does not make the whole application private by construction. A client may still send analytics, crash reports, device identifiers, remote fallback requests or cloud-synced chat history. Web search, email, calendars, MCP servers and other tools necessarily send selected data outside the phone. An embedded HTTP API can expose prompts to the local network if configured carelessly.

Atomic Chat, for example, markets offline prompt processing while its App Store privacy label says device identifiers, product interaction and diagnostics may be collected without being linked to identity. That is not evidence that prompts leave the phone. It is evidence that “local model,” “offline capable” and “zero network activity” are three different claims.

A serious privacy test should download the model, disable remote tools and fallbacks, run in airplane mode, inspect later network traffic, review app analytics settings and test whether chat data is included in backups or sync. Local inference is a strong privacy primitive; the surrounding app determines whether that primitive survives contact with the product.

Which iPhones could plausibly run it?

Apple does not list RAM in its consumer specification pages. The capacities below come from Xcode reporting and independent device tests, not Apple marketing. Apple also does not promise that an app receives a fixed fraction of physical RAM. Its increased-memory-limit entitlement can raise the ceiling on supported devices, but developers must query available memory dynamically and still avoid jetsam.

The practical threshold in today’s evidence is not “an iPhone.” It is an iPhone with at least 8GB of RAM, the right entitlement/runtime, a short context, no drafter, and enough free process budget to keep Metal buffers resident. Twelve-gigabyte Pro hardware offers the safest headroom, but iOS process limits—not the box RAM number alone—remain decisive.

Reproducible independent test plan

A conclusive reproduction should freeze the artifacts and test the claim as a system, not as a single generation clip.

1. Freeze every input. Publish the device identifier, retail model, RAM source, iOS build, battery health, app build, signing entitlement, runtime/backend, compiler, model SHA-256, repository SHAs, context size, KV format, flash-attention setting, sampler and chat template.
2. Test both relevant runtimes. Reproduce PrismML’s MLX Swift path and the public llama.cpp Metal path separately. Pin commits, publish the Xcode project and record whether stock upstream or PrismML forks are required.
3. Measure component-by-component memory. Record process physical footprint, os_proc_available_memory, Metal allocations and jetsam logs for text-only, +vision, +drafter and both. Distinguish download size, resident weights, prefill peak and decode steady state.
4. Use four real contexts. Run 4K, 32K, 100K and 262K prompts with retrieval questions whose answers sit near the beginning, middle and end. Allocation without correct retrieval does not count. Capture first-token latency and the prefill-to-compression transition.
5. Separate burst from sustained speed. Run cold and warm TG128, TG512 and at least 30 minutes of continuous generation, in palindrome order, with five trials. Report median, min/max, thermal state and errors rather than one best clip.
6. Measure energy honestly. Log battery percentage and voltage where available, external power if possible, screen brightness, radio state, ambient temperature, case, battery health and surface temperature. Avoid projecting a full charge from a five-percent slice without labeling the projection.
7. Audit capability by category. Run FP16, Q4, Q2, ternary and one-bit with identical prompts, seeds, temperatures, thinking settings and sample counts. Include reasoning, executable coding, strict instruction constraints, multi-step tool schemas, multilingual factual recall, MMMU/OCR-style vision and long-context retrieval.
8. Test privacy and failure behavior. Repeat after download in airplane mode, capture network activity when connectivity returns, disable remote fallback, record watchdog/Metal failures, and publish raw prompts, outputs, logs and scoring scripts.

The pass condition should be explicit: an independent Pro Max run within 10% of 11.0 tok/s, stable for 30 minutes; 4K peak below the observed process ceiling; no context corruption; and publicly inspectable logs. Long-context claims should be graded separately at each depth.

FAQ

Is Bonsai really a 27B model?

Yes. It preserves the Qwen3.6-27B-derived architecture and approximately 27.3B language weights, represented in binary groups with shared scales. The small file comes from the low-bit representation, not from silently reducing the parameter count.

Is the model only 3.9GB in memory?

No. The live GGUF language file is 3.803GB, but PrismML measures about 5.2GB peak at 4K on llama.cpp and 5.9GB on MLX. Context, runtime buffers, vision and the drafter add memory. The public MLX repository itself is about 5.159GB on disk.

Can users run Bonsai 27B on an iPhone today?

Yes, through third-party apps or by building a developer project. PrismML’s own iPhone documentation is not a one-click runtime: it points to a Swift library fork without a complete app or benchmark harness. The strongest open reproduction requires Xcode, code signing and an increased-memory entitlement.

Has PrismML’s 11 tok/s result been independently reproduced?

Not on the identical iPhone 17 Pro Max and MLX Swift setup. Independent evidence supports the ballpark: 7.82 TG128 and 9.0 tok/s chat on a base iPhone 17, plus a separate 10 tok/s RunAnywhere report on an iPhone 17 Pro.

Can the full 262K context run on an iPhone?

There is no public reproduction. PrismML’s 9.4GB figure is a vendor projection using ideal four-bit cache arithmetic, already above the reported iOS process budget. Live cache overhead raises steady-state estimates, and the documented MLX Swift path holds prefill KV in FP16 before compressing it, potentially making transient peak far larger.

Does one-bit Bonsai retain 89.5% of FP16 quality?

That is PrismML’s ratio of two aggregate benchmark averages. Capability retention is uneven: math loses 3.67 points, while instruction following, tool use and vision lose roughly 13–14 points. The aggregate is useful context, not a reliability guarantee.

Is the vision system one-bit too?

No. The language weights are binary; the separate vision tower uses HQQ four-bit weights. Vision also adds storage and peak memory and was excluded from the 11 tok/s throughput result.

Does local inference guarantee privacy?

No. It can keep model computation and prompts on-device after download, but the app may still use analytics, cloud sync, remote fallback or networked tools. Privacy must be tested at the application and workflow level.

Final assessment

“27B on an iPhone” survives serious scrutiny—but only after shrinking the claim to what the evidence actually proves.

The weights fit. Independent source code and logs show useful short-context generation on an 8GB base iPhone 17. Separate commercial runtime evidence lands near PrismML’s speed. That is a real milestone.

The rest of the headline is aspirational. PrismML’s exact 11 tok/s Pro Max result lacks raw logs and an official reproducible app. Its 3.9GB figure excludes the public MLX package overhead, runtime memory and optional components. Its 9.4GB full-context figure is projected, not measured, and the public MLX cache flow may need a much larger transient prefill peak. Quality is strongest in math and coding, while the exact capabilities most useful to autonomous phone agents—strict instructions, tools and vision—lose far more.

Bonsai 27B is therefore best understood as an extraordinary compression and systems demonstration that has crossed into early practical use, not as a pocket-sized replacement for full-precision Qwen3.6-27B. The headline is justified as an engineering first. It is not yet justified as a description of an all-purpose 27B experience on a phone.

Dated primary sources

Source snapshot checked July 31, 2026 (Pacific Time). Repository and model links are pinned where possible.

PrismML model and runtime sources

• July 14, 2026: PrismML launch article, including the Pro Max cached/prefilled multimodal demo caption.
• July 14, 2026: PrismML press release, the conflicting “iPhone 17 Pro” wording.
• July 14, 2026: Bonsai 27B whitepaper, commit baa67a6, vendor memory, quality, speed and battery methodology.
• July 17, 2026: One-bit GGUF, revision f10afb3, exact files and optional components.
• July 14, 2026: One-bit MLX, revision ef22f23, Apple package, configuration and model card.
• July 18, 2026: Ternary GGUF, revision abbae72.
• July 14, 2026: Ternary MLX, revision 70f75f3.
• July 30, 2026: Bonsai demo repository, revision f904ea2, desktop setup and current four-bit KV notes.
• July 31, 2026: PrismML llama.cpp fork, revision 9ca265a.
• July 14, 2026: PrismML MLX fork, revision 10b5fe4.
• June 11, 2026: PrismML MLX Swift fork, revision e40e0a5.

Independent and platform sources

• July 17, 2026: Independent Japanese iPhone investigation and source/log repository, revision 48f4377.
• July 19, 2026: RunAnywhere’s iPhone 17 Pro report, including its 10 tok/s result and public benchmark client.
• July 2026: Independent Jetson comparison repository, a small custom capability check.
• Current snapshot: MLX Swift LM KV-cache documentation, revision cd1ab3d, documenting FP16 prefill and 3.56× affine-four compression.
• Current snapshot: Apple increased-memory-limit entitlement documentation and jetsam guidance.
• September 9, 2025: Xcode-reported iPhone 17 RAM capacities: 8GB base, 12GB Air/Pro/Pro Max. Apple does not advertise these capacities.
• September 9, 2025: Apple’s iPhone 17 Pro/Max announcement, A19 Pro and vapor-chamber cooling.
• Current snapshot: Atomic Chat App Store listing and privacy label.