{"id":48117,"date":"2025-06-17T16:16:08","date_gmt":"2025-06-17T20:16:08","guid":{"rendered":"https:\/\/mjtsai.com\/blog\/?p=48117"},"modified":"2025-07-23T13:07:16","modified_gmt":"2025-07-23T17:07:16","slug":"foundation-models-framework","status":"publish","type":"post","link":"https:\/\/mjtsai.com\/blog\/2025\/06\/17\/foundation-models-framework\/","title":{"rendered":"Foundation Models Framework"},"content":{"rendered":"

Apple<\/a> (MacRumors<\/a>, 9to5Mac<\/a>, Hacker News<\/a>, Slashdot<\/a>):<\/p>\n

\n

Apple is opening up access for any app to tap directly into the on-device foundation model at the core of Apple Intelligence.<\/p>\n

With the Foundation Models framework, app developers will be able to build on Apple Intelligence to bring users new experiences that are intelligent, available when they’re offline, and that protect their privacy, using AI inference that is free of cost. For example, an education app can use the on-device model to generate a personalized quiz from a user’s notes, without any cloud API costs, or an outdoors app can add natural language search capabilities that work even when the user is offline.<\/p>\n

The framework has native support for Swift, so app developers can easily access the Apple Intelligence model with as few as three lines of code. Guided generation, tool calling, and more are all built into the framework, making it easier than ever to implement generative capabilities right into a developer’s existing app.<\/p>\n<\/blockquote>\n\n

There are two WWDC<\/a> sessions<\/a> and documentation<\/a>.<\/p>\n\n

Daniel Jalkut<\/a>:<\/p>\n

\n

This is EXACTLY (the bare minimum) what developers have been asking for!<\/p>\n<\/blockquote>\n\n

mxdalloway<\/a>:<\/p>\n

In my opinion this is revolutionary.<\/p>

It was obvious that we would get framework access to models eventually, but I’m a little shocked that it’s already here.<\/p>

I was skeptical of the performance in the demos, but running on M1 MBP I’m happy with the performance.<\/p>

@Generable<\/code> macro is intuitive to use and so far I’m impressed with the quality of the structured results that the model generates (admittedly, I need to do more extensive testing here but first impressions are promising).<\/p><\/blockquote>\n\n

Steve Troughton-Smith<\/a>:<\/p>\n

The wider tech press seems to think that Apple failed to show anything meaningful to do with AI at WWDC, without understanding that access to the Foundation Models<\/em> is bigger<\/em> than anything Apple announced at last year’s WWDC with Apple Intelligence. It’s what will give a million apps new AI features, and it’s built-in, and free.<\/p>

As much as I want Siri to not suck, I have ChatGPT on all my devices, and that solves 95% of the use-cases I have.<\/p><\/blockquote>\n\n

Drew McCormack<\/a>:<\/p>\n

\n

Having played with the new Foundation Models framework and thought about ways we can use it in our apps, I think it could be Apple’s Trojan horse for AI. It barely gets a mention in mainstream media, understandably, but it leverages Apple’s developer base. I think we are going to see very creative uses in apps, and Apple just have to iterate year on year (eg add private cloud compute next year).<\/p>\n<\/blockquote>\n\n

Drew McCormack<\/a>:<\/p>\n

Was optimistic about Foundation Models yesterday, and today I think I know why they didn’t ship the improved Siri. The local model really is pretty thick. I thought it would be capable of stringing together tool calls in a logical way, and sometimes it is, but other times it fails to understand. Exactly the same prompt will work one time, and fail the next. Sounds like what Apple was saying about the new Siri.<\/p><\/blockquote>\n\n

Kuba Suder<\/a>:<\/p>\n

\n

Ahh wait, so the Foundation Models thing will only work on the latest and greatest phones, right? 🫤<\/p>\n<\/blockquote>\n\n

It doesn’t fall back on Private Cloud Compute.<\/p>\n\n

Steve Troughton-Smith<\/a>:<\/p>\n

\n

Oof, the FoundationModels framework is not exported to Mac Catalyst in Xcode 26 seed 1 😫 That puts a damper on prototyping<\/p>\n<\/blockquote>\n\n

Matt Gallagher<\/a>:<\/p>\n

The Foundation Models in macOS 26 are quantized to 2 bits? I’m amazed anything coherent comes out.<\/p><\/blockquote>\n\n

Peter Steinberger<\/a> (Reddit<\/a>):<\/p>\n

\n

Apple’s on-device AI has a brutally small 4,096 tokens window.<\/p>\n<\/blockquote>\n\n

Jordan Morgan<\/a>:<\/p>\n

In this post, I’ll show you the basics of how to get started with the Foundation Models framework, and we’ll even make a few calls to its API to generate some responses.<\/p><\/blockquote>\n\n

Previously:<\/p>\n