The Emergence of Bonsai: A 27-Billion Parameter Model on iPhones
The landscape of artificial intelligence (AI) is rapidly evolving, necessitating powerful models to run locally on devices. PrismML emphasizes the importance of local execution for AI models, given the complexity of modern AI applications. Unlike traditional approaches where a single model invocation occurs, today’s agents execute hundreds of sequential model calls, each influenced by contextual inputs and leading to structured outputs.
Cost and Latency Issues with Cloud-Based AI
Operating on the cloud might seem convenient, yet it comes with significant drawbacks. Each token incurs costs, every step introduces network latency, and sensitive data, including screen contents and private documents, might leave the device during these processes. Running an AI model directly on a device eliminates these issues, leading to zero marginal costs for each cycle and keeping user data entirely local. PrismML sees this as the foundation for consistently operating agents within devices, enabling local solutions for simple tasks while offloading more complex tasks to cloud-based frontier models.
Collaboration with Apple and Efforts in Compression Technology
Recent reports indicate that PrismML is in talks with Apple regarding its underlying compression technology. According to PrismML’s CEO Babak Hassibi, Apple and other tech companies are currently testing the models for speed, energy consumption, and performance. While these discussions are still in their early stages, they are progressing positively.
Two Variants for Laptop and Smartphone
Standard models of this scale typically occupy around 54 GB of storage, and even with compression techniques, they reduce to about 18 GB. PrismML offers two significantly smaller variants: the quality-oriented version at about 5.9 GB for laptops, and a smaller variant designed for smartphones, weighing in at approximately 3.9 GB. This smaller size makes it feasible for the iPhone 17 Pro Max, which has limited storage availability given that an iPhone with 12 GB of RAM can allocate roughly 6 GB to a single app.
PrismML employs a novel approach to neural network weight storage, utilizing only one or nearly two bits per weight instead of the conventional 16 bits. In their most aggressive variant, weights recognize only two states, while slightly larger versions accommodate three states. This innovative technique pervades the entire language model.
Quality Assessment and Benchmarking Results
PrismML conducted an evaluation across 15 benchmarks, revealing that the larger variant retains 95% of the original model’s performance, while the smaller version maintains around 90%. Key functionalities like mathematics and coding remain largely unaffected. However, more aggressive compressions exhibit significant performance losses, particularly in tasks involving image recognition, instruction following, and tool utilization. For instance, a highly compressed Qwen3.6-27B model, despite occupying 9.4 GB, scores a mere 72.7, while the smaller Bonsai model achieves 76.1.
On an iPhone 17 Pro Max, the smaller variant can generate around 11 tokens per second. A battery test indicated approximately 672 generated tokens per percentage of battery charge, which translates to about 67,000 tokens for a full charge, albeit with a slight throttle after five minutes of operation.
Apple’s Potential Leap in Local AI Solutions
The model weights are available under the Apache-2.0 license, making Bonsai 27B operable on Apple devices through the MLX framework as well as NVIDIA GPUs. Additionally, PrismML offers a limited-time free developer-preview API and a live demo via HuggingFace.
Founded with support from Khosla Ventures, Cerberus, and Google, PrismML has its eyes set on applying their technology to Google’s Gemma model series, which already operates efficiently on smartphones. For Apple, acquiring a licensed compression technique is crucial, especially as their in-house models have fallen short in recent benchmarks. The latest revelations at WWDC 2026 showcased an updated Siri version, deemed foundational through models developed in collaboration with Google based on Gemini technology.
In summary, with Bonsai and its local execution capability, Apple appears poised to strengthen its position in the rapidly advancing AI field, potentially transforming user experiences on devices like the iPhone 17 Pro Max and beyond.
