Bonsai 27B: The First 27B-Class Model That Runs on a Phone

PrismML just announced Bonsai 27B — the first 27-billion-parameter language model that runs on a phone. Not a stripped-down demo, not a proof-of-concept. A full multimodal model with reasoning, coding, and agentic workflow support, compressed from 54GB to 3.9GB while retaining 90% of full-precision performance.

This is a genuine Pareto shift: bringing 27B-class capability — previously confined to cloud GPUs — onto personal devices.

From 54GB to 3.9GB: What Bit Quantization Actually Achieves

A 27B-parameter model at 16-bit precision occupies roughly 54GB — far beyond any smartphone or mainstream laptop. Even a good 4-bit build, at 18GB, is still too large for a phone.

Bonsai 27B uses end-to-end low-bit quantization — not just a few layers selectively, but the entire network: embeddings, attention, MLPs, and the LM head, with no higher-precision escape hatches. The results:

Variant	Bits/weight	Size	Benchmark vs FP16	Runs on
Ternary (1.58-bit)	1.71 effective	5.9 GB	95%	Mainstream laptop
1-bit Binary	1.125 effective	3.9 GB	90%	iPhone 17 Pro

The 1-bit variant achieves 11 tokens/s on an iPhone 17 Pro — readable speed, sufficient for local agentic workflows. For comparison, the most aggressive conventional 4-bit build of the same base model scores lower than 1-bit Bonsai while consuming 2.5x more memory.

The key metric isn't the bit count — it's intelligence density. PrismML defines this as benchmark score per GB of memory. 1-bit Bonsai 27B delivers 0.53 per GB — more than 10x the full-precision baseline and roughly 2.7x the best competing low-bit alternative.

For competitive context: Google's Gemma 3 (27B) requires server-class GPUs, Microsoft's Phi-4 (14B) still needs at least 8GB VRAM, and Apple Intelligence runs on-device but uses much smaller models (~3B). Bonsai 27B is the first to bring genuine 27B-class capability to a mobile device.

Built for Work, Not Just Chat

What sets Bonsai 27B apart from typical local models: it's not a lightweight chatbot. PrismML built this for agentic workloads — multi-step reasoning, structured tool calling, computer-use loops that stay coherent across hundreds of steps.

Across 15 benchmarks (math, coding, tool calling, instruction following, vision, knowledge):

• Math: Ternary retains 93.4% (from 95.3% FP16), 1-bit retains 91.7%
• Coding: Ternary 86.0%, 1-bit 81.9% (HumanEval+, MBPP+, LiveCodeBench)
• Agentic & Tool-calling: Ternary 74.0%, 1-bit 66.0% (BFCL v3, TauBench)

Math and coding are nearly untouched by quantization — exactly the capabilities agentic workloads depend on. This isn't accidental: mathematical reasoning and code generation rely on discrete, highly structured patterns, so they tolerate weight rounding well. In contrast, continuous-perception tasks like image understanding and complex instruction following require higher per-weight precision — which is why vision and instruction following drop more noticeably.

The model carries a full 262K-token context window and supports speculative decoding. It's multimodal — the vision tower ships in compact 4-bit form, so on-device workflows can process screenshots, documents, and camera input.

Why Developers Should Care

Over the past year, the agentic AI trend has changed how models are used: no longer one API call → one answer, but hundreds of sequential calls, each carrying context, producing structured output, and feeding the next step.

In this pattern, API cost and latency become the bottleneck. Cloud models remain essential for the hardest tasks, but a local agent — private, zero network latency, no rate limits — unlocks use cases that were previously impossible:

• CI/CD agent on your dev machine: An agent running Bonsai 27B sits in your repository, auto-reviews every commit, writes unit tests for new code, catches bugs before you push — all without sending a single line of code to the cloud.
• On-device assistant for sensitive work: Review contracts, analyze internal documents, process screenshots containing customer data — everything happens on-device, no data leaves the machine.
• Edge deployment: Applications requiring complex reasoning where internet is unreliable — factories, ships, rural clinics.

The licensing matters too: Bonsai 27B is released under Apache 2.0 — commercial use allowed, no restrictions. Compared to other on-device models locked behind proprietary licenses or API-only access, this is a fully open option.

The Real-World Caveats

A clear-eyed view is necessary. Vision capabilities take a significant hit — ternary retains 65.2% and 1-bit only 59.6% on MMMU Pro and OCRBench. Instruction following also drops notably (ternary 71.8%, 1-bit 65.8%).

Why the uneven impact? Quantization rounds weights to discrete values, and tasks requiring fine per-weight precision suffer most. Detailed image understanding depends on every pixel-level feature being exact. Complex instruction following requires weight-level precision to preserve subtle semantic distinctions. Math and code, by contrast, have higher structural redundancy — the "shape" of the computation survives rounding much better.

For tasks requiring complex instruction adherence or detailed image understanding, cloud models remain the better choice. Bonsai 27B doesn't replace Claude or GPT — it expands the frontier of what can run locally.

Additionally, while PrismML demonstrated agentic loops on an iPhone 17 Pro, they noted the phone demo runs with "cached and prefilled image context" — not live end-to-end. Real-world performance in complex workflows still needs community verification.

Takeaways for Developers

• 27B-class AI is now on a phone — Ternary retains 95% of benchmarks, 1-bit runs on iPhone 17 Pro at 11 tokens/s
• Apache 2.0 license — commercial use, no restrictions, integrate freely
• Agentic-ready: tool calling, 262K context, speculative decoding, multimodal (text + vision)
• Not a silver bullet: vision and instruction following still lag cloud models significantly — pick the right tasks
• Intelligence density > parameter count: a new metric matters more than raw size — Bonsai 27B is 10x more efficient per GB than the full-precision baseline

A 27B model on a phone is no longer theoretical. The question now isn't "can it run?" — it's "what will you build with it?"

---

Content assisted by AI (Amy 🌸). Reviewed by the author.