Apple eyes startup that shrinks AI models to run on iPhones

Apple is in early talks with PrismML, a Silicon Valley startup backed by Khosla Ventures and spun out of the California Institute of Technology, about technology that shrinks large AI models small enough to run directly on iPhones. PrismML's CEO Babak Hassibi confirmed to CNBC that Apple and other companies have been evaluating the startup's models while measuring their speed, energy efficiency and performance on devices. He characterized the discussions as very early but said "things are progressing nicely," though it remains unclear where the conversations will lead. Apple declined to comment.

On Tuesday, PrismML publicly released compressed versions of Alibaba's open-source Qwen model, which the company reduced from roughly 54 GB to less than 4 GB, allowing all 27 billion of its parameters to run on an iPhone 15 or newer. The startup achieves this compression by drastically simplifying how a model's internal information is stored—reducing each value from 16 bits to just one or three possible values, which significantly cuts the memory required to store and operate the model. Hassibi compared the approach to the chip industry's move from eight-bit to four-bit computing, but taking it a step further.

According to PrismML's testing, the compressed models use between 10 and 15 times less memory, generate responses six to eight times faster and consume three to six times less energy than conventional versions running on existing hardware. However, Hassibi acknowledged a trade-off: the compressed models typically lose a few percentage points of overall performance, with factual recall weakening before skills such as reasoning, math and coding. The startup is releasing two compressed versions for free, designed to run on everyday devices including iPhones, MacBooks and Nvidia-powered PCs. Hassibi said Google's open-source Gemma model is next in the pipeline, followed by much larger models from frontier labs that currently require datacenter hardware. The underlying patents came from Hassibi's research group at Caltech, and the university licenses them exclusively to PrismML. In March, the company raised a $16.25 million（約26億円） seed round backed by Khosla Ventures and other investors.

The timing of the release is significant. It comes one day after Apple opened the public beta of iOS 27, giving iPhone owners their first broad access to the company's long-delayed overhaul of Siri. Apple is trying to make Siri more competitive with assistants from OpenAI and Anthropic while keeping more personal information and AI processing on the device. By running more capable AI directly on iPhones, Apple could reduce latency associated with sending data to remote servers, lower cloud-computing costs and strengthen its privacy pitch. It would also allow certain features to work without an internet connection. Carolina Milanesi, president and principal analyst at Creative Strategies, said smaller models could let Apple move more demanding features onto the iPhone, including computational photography, video generation and health or fitness tools that rely on sensitive personal data. "The more you can do on device, the better it is," she said, pointing to health and medication data that users would want to keep private.

Apple already runs parts of its AI system locally, including translation, some summarization and features tied closely to personal information, with more complex requests routed to Apple's private cloud infrastructure or outside models. Horace Dediu, founder of Asymco, said Apple is likely trying to keep the large majority of common Siri interactions on-device while reserving the most demanding tasks for the cloud, giving Apple lower latency, greater privacy and potentially lower licensing and cloud costs. Apple may also have an advantage because it designs the iPhone's chips and software together, giving it tighter control over how AI runs on the device.

But analysts cautioned that PrismML's claims still need to be proven outside controlled demonstrations. Tarun Pathak at Counterpoint Research said the model's performance on lengthy prompts, battery consumption during multitasking and reliability across millions of requests will be critical: "The ultimate test will be millions of queries, thousands of device combinations and robust testing at scale." Phil Solis at IDC flagged power consumption as potentially the biggest open question, noting that a model capable enough to be used frequently—or continuously in the background for agent-like tasks—could drain a phone's battery even if it requires less memory.

PrismML's release also touched off debate over whether AI efficiency improvements could eventually reduce demand for memory chips. Morgan Stanley estimates Apple's average DRAM (dynamic random access memory) cost per bit could rise roughly 190% year over year in fiscal 2027, with NAND costs up about 180%, and expects Apple to raise the starting price of comparable iPhone 18 models by about $200 to protect margins. PrismML said its approach could allow a cloud model that normally requires eight GPUs to run on one, while also allowing models that once required a server to move onto phones and laptops. However, Gil Luria at D.A. Davidson argued that shrinking models would not eliminate the need for processors or memory—it could simply move more of those chips from datacenters into phones and other devices. "It's not that you're not going to need the chip," Luria said. "You're still going to need the GPU, and you're still going to need the memory." He also noted that running AI on individual devices can actually be less efficient than using shared datacenter infrastructure because chips in phones may sit idle much of the time, and efficiency breakthroughs can lead to more use rather than lower spending.

Apple's conversations with PrismML reflect a fundamental challenge facing the company's AI strategy: the most capable models typically require too much memory and processing power to run on a smartphone. By moving more inference (the step where an AI produces an answer) onto the device itself, Apple could address multiple constraints at once—lower latency for responses, reduced reliance on cloud infrastructure and its associated costs, and stronger privacy protection for sensitive health and personal data that users would prefer to keep local. This aligns with Apple's broader effort to make Siri competitive with assistants from OpenAI and Anthropic while leveraging the tight integration between iPhone hardware and software that has long been a company strength.

However, analysts have flagged critical unknowns. Tarun Pathak at Counterpoint Research emphasized that "the ultimate test will be millions of queries, thousands of device combinations and robust testing at scale," pointing to concerns about the model's performance on lengthy prompts, battery consumption during multitasking, and reliability across millions of real-world requests. Phil Solis at IDC flagged power consumption as potentially the biggest open question—a model capable enough to run frequently or continuously in the background could still drain a phone's battery even if it uses less memory than datacenter alternatives.

The release also touches a nerve in the chip industry. PrismML's claims that its approach could allow a cloud model normally requiring eight GPUs to run on one, or move models from servers onto phones, raised immediate questions about whether AI efficiency gains might eventually reduce demand for memory chips and expensive infrastructure. Yet analysts like Gil Luria at D.A. Davidson argued that shrinking models would not eliminate chip demand—it would simply move more processors and memory from datacenters into phones and other devices. He also noted that on-device AI can be less efficient overall than shared datacenter infrastructure because phone chips may sit idle much of the time.

No comments yet. Be the first to share your thoughts!