Bonsai 27B: First 27B AI Model to Run on a Phone
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- PrismML launched Bonsai 27B, based on Qwen3.6 27B, as the first 27B-class model to run on a phone — with binary 1-bit weights packing the model into 3.9 GB, small enough to clear the ~6 GB memory budget available to an app on a 12 GB iPhone 17 Pro.
- A ternary variant (5.9 GB, 1.71 effective bits per weight) targets laptops and retains 95% of full-precision baseline scores; the 1-bit version retains 90% across a 15-benchmark suite covering knowledge, reasoning, math, coding, instruction following, tool calling, and vision.
- 1-bit Bonsai 27B hits 0.53 intelligence density per GB — more than 10x the full-precision baseline and roughly 2.7x the best conventional low-bit alternative, which uses 2.5x more memory and still scores lower.
- The model reaches up to 163 tok/s in 1-bit and 134 tok/s in ternary on an NVIDIA RTX 5090, and 87 tok/s / 58 tok/s respectively on Apple's M5 Max.
- Bonsai 27B ships with a full 262K-token context, a compact 4-bit vision tower for screenshots and camera input, speculative decoding for lossless acceleration, and an Apache 2.0 license — with no higher-precision escape hatches anywhere in the language network.
- PrismML emerged from Caltech researchers and is backed by Khosla Ventures, Cerberus, Google, and Samsung; the company frames the release as enabling hybrid agentic architectures that route non-frontier and privacy-sensitive tasks on-device.
Why it matters: Agentic AI workloads run hundreds of sequential model calls, each carrying context and intermediate data — every step a cloud API call adds per-token cost and ships user files, screen content, and tool results across the network. At 3.9 GB the 1-bit Bonsai 27B is the first 27B-class model whose footprint collapses that equation: a model capable of sustained multi-step tool use and vision can now run locally, where the marginal cost of a hundred-step loop is zero and private data stays on the device by construction.
