PrismML's Bonsai 27B: Full reasoning model compressed to fit iPhone

PrismML, backed by Khosla Ventures, Cerberus, and Google, with ongoing support from Samsung, has unveiled Bonsai 27B, a compressed version of the Qwen3.6-27B model designed to run efficiently on consumer devices. The central innovation is a weight quantization approach: instead of storing each neural network parameter as a 16-bit number, PrismML compresses weights to 1–2 bits per parameter across the entire language model. The company offers two versions: a quality-focused variant at about 5.9 GB (designed for laptops, though shipped packages may be larger depending on the runtime environment—7.2 GB for the llama.cpp version and 8.49 GB for the MLX version), and a smaller variant at about 3.9 GB that fits on an iPhone 17 Pro Max despite the device's storage constraints.

According to PrismML's evaluation across 15 benchmarks, the larger 5.9 GB variant retains 95 percent of the original Qwen3.6-27B model's performance, while the smaller 3.9 GB variant keeps 90 percent. Math and coding tasks were "virtually unaffected." Larger performance drops appeared in image understanding, instruction following, and tool use, particularly under the most aggressive 1-bit compression. By contrast, a conventionally compressed Qwen3.6-27B model at 9.4 GB scores only 72.7 points on one benchmark, while the smaller Bonsai variant at 3.9 GB scores 76.1. On an iPhone 17 Pro Max, Bonsai 27B generates about 11 tokens per second. In battery testing, the model produced roughly 672 generated tokens per percentage point of battery charge, suggesting approximately 67,000 tokens per full charge, though the chip throttled slightly after about five minutes of continuous use.

PrismML's motivation is rooted in the economics and privacy of on-device inference. Modern AI agents may make hundreds of model calls in sequence, each with context and producing structured output that feeds the next step. Running these loops in the cloud accumulates per-token costs, introduces network latency, and exposes intermediate results, tool calls, and private data—such as screen content or documents—to cloud servers. On-device execution cuts marginal loop costs to zero and keeps data local. According to a CNBC report, PrismML is already in talks with Apple about the compression technology. CEO Babak Hassibi confirmed that Apple and other companies are testing the models for speed, power consumption, and performance, describing the talks as "very early" but noting that "things are progressing nicely."

The release aligns with Apple's stated need for stronger on-device capabilities. At WWDC 2026, Apple unveiled a revamped Siri built on foundation models developed in partnership with Google using Gemini technology. Apple's most powerful on-device model currently requires an iPhone with at least 12 GB of RAM, and complex queries are offloaded to NVIDIA GPUs in Apple's cloud infrastructure. A licensed compression technology could enable Apple to run more capable models locally without expanding device requirements, narrowing what the body describes as Apple's local AI performance gap. PrismML plans to next apply its compression approach to Google's Gemma model series, smaller versions of which already run on smartphones. The Bonsai 27B model weights are available under the Apache 2.0 license, and the company offers a time-limited free Developer Preview API and a live demo on HuggingFace. Bonsai runs on Apple devices via Apple's MLX framework and on NVIDIA GPUs.

PrismML's release of Bonsai 27B addresses a real constraint in modern AI applications: running models in the cloud accumulates per-token costs and latency with every call, especially when an agent must make hundreds of sequential calls. By moving inference to the device, the marginal cost of each loop drops to zero, and sensitive intermediate data—screen content, documents, tool calls—never leave the user's device. This is the foundation PrismML sees for always-on assistants and privacy-preserving hybrid systems where simple tasks stay local and only hard problems are sent to frontier cloud models.

The compression technique—reducing weights from 16 bits to 1–2 bits across the entire model—is aggressive, yet PrismML's benchmarks show it preserves 90–95 percent of the original Qwen3.6-27B model's performance. Math and coding tasks are nearly unaffected, though image understanding and instruction following show larger drops under the most extreme compression. Apple's interest in licensing this technology is significant: Apple's own on-device models have trailed competitors in performance, and its current Siri implementation requires at least 12 GB of RAM for the most powerful on-device model, offloading complex queries to cloud GPUs. A licensed compression layer could allow Apple to run more capable models locally, narrowing that performance gap while keeping user data private.