Apple Foundation Models in appleOS 27
At the heart of this architecture is our third generation of Apple Foundation Models (AFM), a family of five foundation models custom-built in collaboration with Google. These span from on-device models to server-based models running on Private Cloud Compute.
Apple Foundation Models are built to unlock a wide range of helpful experiences for our users, like an entirely new Siri and intelligent tools that make everyday apps smarter and more useful.
“The amount of the Google Assistant we use is none,” Federighi said, explaining that Apple uses none of the Gemini models deployed to Google’s customers, none of Google’s client-side code, and no Google Search infrastructure as the knowledge backbone.
The cloud models hosted on Private Cloud Compute are interesting: they’re not “Gemini” models; they’re Apple Foundation Models trained using proprietary data via RL, which were then “refined” with data from Google’s “frontier” models.
[…]
I’ve been covering the AFM family of models for a while now, and AFM Core Advanced is one of the most interesting on-device models I’ve read about in a while, especially in the context of model size for mobile devices and built-in multimodality with support for text, images, and audio. I’m very keen to play around with this model and understand how it holds up in practice. I wonder if the new CLI (!) for AFM will let you test this one.
It’s very cool that Apple shipped a 20B parameter on-device.
You can’t put 20B parameters in RAM at any reasonable precision. To make it work they are using pretty exotic architecture by today’s standards.
A small model predicts from the query (or prompt) which experts to load from Nand into RAM. The key distinction from a typical MoE is that you do this once per query and then generate all the tokens with the same experts (instead of switching the experts for every token).
Looks like there’s still a 4K context size on local FoundationModels. Pretty limiting.
Really disappointed SwiftUI’s result builder syntax has infected FoundationModels API too.
How did a million modifier methods become the default way to build APIs?
It’s horrible for discoverability, code completion, documentation.
FoundationModels is one of the highlights of this WWDC; I think it’s now finally what we hoped we were getting two years ago, and thought we were getting last year (just the models themselves were barely up for it then). I finally feel like I can build features with it, though I really don’t like the Sword of Damocles Apple is holding above Private Cloud Compute and how it can permanently lock you out of using the models forever across all your apps if you hit a download threshold. Poison pill
These strict limits don’t seem to be getting as much attention as they should. It’s nice that for small developers who meet the above criteria, access to PCC has no cost. But there’s no way (yet?) to buy your way out of these limits.
[…]
The “fewer than 2 million first-time app downloads from any of their apps” restriction is particularly notable. It’s not 2 million installations for apps that are using PCC, but 2 million downloads for any app the developer has ever released.
Is there a real time count somewhere of total lifetime downloads for a developer account, so one can plan if they at risk of the switch tripping? Reminds me of the small business qualifications. I had to try to figure out if an employers developer account was going to exceed the revenue threshold for the year. I was left trying to determine it from sales reports, not to mention sales date versus pay out date confusion. I think I botched the calculation in the end.
Claude for Foundation Models is a Swift package that makes Claude available as a server-side language model in Apple’s Foundation Models framework. The package conforms Claude to the framework’s
LanguageModelprotocol, so you drive it with the sameLanguageModelSessionAPI you use for Apple’s on-device model:respond(to:), streaming, guided generation, and tool calling all work the same way.Requests go directly from your app to the Claude API; Apple is not in the request path and does not see prompts or responses. Usage is billed to your Anthropic account at standard API pricing.
Previously:
- Apple Intelligence in appleOS 27
- Siri AI Announced
- OmniFocus 4.8.10
- OmniOutliner 6.1
- Foundation Models Framework
- Apple Intelligence Foundation Language Models
