Apple Intelligence Announced

Wow. It's disgusting to see Apple fall in the generative AI train this way. Really disrespectful to artists and writers, not to mention the fact that this is going to cause massive headaches for teachers and professors

Yikes

I was hoping that "privacy" would have meant they would run the LLM locally. But no, Apple's definition of privacy is don't trust other corporations running stuff on their servers, but trust us. We promise you, and third parties will vouch, that your data is safe with us.

I'm sure that's fine for everyone who already drinks the "trust Apple" cool-aid. But since governments can force any corporation to silently snitch on its users, trusting anything over which one has no control is already letting the cat out of the bag.

This is currently just a big branding exercise. But given the way they're calling it "Apple Intelligence", they won't be able to give up on it, if OpenAI starts making their lives difficult. This is the first time in a long time that Apple has given up its control over a core technology. They dumped Nvidia and Intel over their lack of control. Now they've given control to OpenAI, with respect to a technology that might well matter more than they do in the long run. Reminds me of Microsoft and IBM...

And as Manx says, their original user base might not be best pleased with this move.

Siri is now officially known as Sam, Scraping All your Memories.
Siri is now just a wrapper around OpenAI.

Generative garbage as far as the eye can see, coming this fall. Relax, it’s iPhone(TM).

Oh, look, Apple already forwards data from apps on your phone to the government. Oh, and they didn't tell us because the government told them not to. But, "trust us", "we're different".

Apple says you have the right to opt out of your website being used to train their LLM, but when you go to the relevant page, there doesn't seem to be any LLM specific commands -- so it seems that you're also opting out of your webpage being found using their search.

@Old Unix Geek Yep, blogged here.

The race to have a statistical model fuck up 20% of what you want to do on your phone is on.

So far Siri had big problems when I mixed German and English. I wonder if this will get better.

If there is an option to use ChatGPT directly why is the hardware minimum a T2 chip for Intel which is only security related?

Yeah, emojis everywhere. Why call them emojis when they are just images of something? I can already create AI icons in Logoist - not very good ones, though.

I too was surprised at Apple limiting these features to the A17 Pro iPhones, despite how much they have been talking up the NPUs on their chips for so long.

My guess is it's a RAM constraint rather than an older-model NPU constraint: 8GB system RAM is probably the absolute minimum for running these generative models. If only Apple weren't so stingy with RAM...

Also, I wonder if we're about to see even more swapping and performance issues on the already-constrained 8GB Macs.

@OUG Yup. My thought too, in the end it's just not private if it's going to Apple. I doubt there is any interest over there in giving control over that or any other connections any time soon either. This is just how it works in the Appleverse now: "Privacy" means "Privacy from anyone but Apple." I also have serious doubts about the quality of the research that will be possible if Apple gates access, because notwithstanding the dangers of security by obscurity, the kind of money that will buy exploits is also very well placed to do that research just fine without Apple. It's all very sad.

@Beatrix Willius Yes, as a T2 iMac owner, I do hope you can still use ChatGPT even without the Apple Intelligence features requiring their models and Silicon. Otherwise they'd just be gating access to a UI, which is certainly plausible for Apple, but it would be very shitty.

"This is the first time in a long time that Apple has given up its control over a core technology"

Apple is working on its own LLMs. It's just that as of right now, the only models that are remotely reliable are OpenAI's. They're going to do an Apple Maps on this as soon as they think people will accept their inferior proprietary option.

I'm curious whether textual input can be used on macOS.

Because the microphone is such a poor input when working in open spaces at offices.

ChatGPT is optional, and every command sent to ChatGPT will have to be manually allowed.
First it will try the built-in LLM, and the Apple's LLM on Apple's server, and if you want you can send things to ChatGPT or other things.

It's the RAM that's the bottleneck. And if you have been following the developments in the LLM space, it has been clear for years, that that would be the case.

My prediction is, that the next flagship will have even more RAM to support future even larger models on-device.

At first, AI seemed great, but then it was indeed awkward to see the other company's system so deeply integrated into Apple's ecosystem. I wondered if Apple had acquired OpenAI or if they were planning to do so... Or maybe they will add more services in the future, such as Gemini.

I think it must be the RAM limitation:
Xcode 16 includes predictive code completion, powered by a machine learning model specifically trained for Swift and Apple SDKs. Predictive code completion requires a Mac with Apple silicon and 16GB of unified memory, running macOS 15. (116310768)

Yes models are big, but there is something very odd about Apple's strategy: spend a lot of silicon on NPUs but not enough on commodity RAM. You could argue that they weren't sure that LLM's would take off, but then why bother spend the money to design and ship NPUs. Even if you're pad limited, you could use the area to boost cache. Adding RAM on the other hand is cheap, and would be valuable for many other uses, even if LLM's didn't take off.

Alternatively, you could speed up the Flash path, and stream chunks of the model into RAM: embeddings are propagated through a stack of transformers, and although there's a 50-100x latency to access Flash, the access pattern is constant and known and you can request it transferred to RAM beforehand using double-buffering. If Flash is still too slow, you can add a large cache to the Flash controller if you feel HBM ram is too expensive.

So basically if you really care about privacy, there are at least 2 ways to attack the problem to make a relatively large model work on device. It might be a bit slower than you'd like, but you can maintain your privacy selling point. So I don't find the "It's the RAM, you ignoramus, even morons could see that coming decades ago!" argument particularly convincing for a company that owns the whole stack from the hardware up.

Yes, Apple is making its own LLM on device and off (and they're using TPUs from Google and GPUs to train it, not Apple Silicon, just saying). We'll see whether it ends up replacing ChatGPT though. Sounds to me like their AI department is currently seriously outgunned.

I also find it odd that they go with OpenAI, given its tight partnership with Microsoft. You'd have thought that Anthropic might be a more likely choice. That suggests to me that perhaps that OpenAI showed more impressive demos of future products than their competitors did.

@OUG
> I also find it odd that they go with OpenAI, given its tight partnership with Microsoft. You'd have thought that Anthropic might be a more likely choice. That suggests to me that perhaps that OpenAI showed more impressive demos of future products than their competitors did.

I think most likely the latter, as you touched upon, their own AI seems to seriously lag behind. Should they pick a seemingly inferior model, the gap between the competitors would widen. So I think they are desperate right now, until they get their own act together in the AI department, which I have an impression of being in a kind of panic mode.

@old Unix geek the support doc you linked does say you can opt out of just AI training and still be indexed for search.

“You can add a rule in robots.txt to disallow Applebot-Extended, as follows:
User-agent: Applebot-Extended
Disallow: /private/
Applebot-Extended does not crawl webpages. Webpages that disallow Applebot-Extended can still be included in search results. Applebot-Extended is only used to determine how to use the data crawled by the Applebot user agent.
Allowing Applebot-Extended will help improve the capabilities and quality of Apple’s generative AI models over time.”

Here is a good thread from Matthew Green about Private Cloud.

https://ioc.exchange/@matthew_d_green/112597849837858606

The improved image search feature is tempting but otherwise I have no idea why someone would want any of these features, or be willing to contribute to the intellectual theft and environmental damage this will do. How does this square with Apple’s environmental goals? Do they not count what they farm out to Open AI as contributing to their carbon footprint? Even without counting Open AI their own server infrastructure will go up significantly.

@Eric: I missed that. Thanks!

I'll admit that I have very little knowledge of the nuts-and-bolts of how LLMs work. Just from imaging the amount of data involved, it seems like a "general knowledge" LLM would be too big to house entirely on-device. If they really are that big, then it might not matter if your device has 8GB or 80GB of RAM.

Also, I think it was a *brilliant*, if somewhat obvious, move for Apple to call their artificial intelligence system Apple Intelligence. Apple can now use Apple Intelligence and AI interchangeably if they like, while competitors may feel compelled to say "artificial intelligence" (spelled out) so they don't have people thinking about Apple (even subconsciously) when they say AI.

I found it a bit short-sighted to put their name on something that will do stupid stuff.

When Gemini fucks up there is no quick way to make a catchy headline about Goggles lack of intelligence.

With Apple they will literally write themselves.

@Kristoffer: Maybe. But Apple is doing a couple of smart things with this. First, they've already said that it's all going to be labeled "Beta" when it first comes out, even in the fall. That gives them a little wiggle room to get things right. Second, they're putting gates around the OpenAI part by asking permission to engage with it. Sure, from a privacy standpoint it lets the user know what's going on with their data. But it's also a demarcation of functionality, clearly showing the user that they're stepping outside the walled garden.

To be honest, I don't think the "gates around OpenAI" will make as much difference as you think, because I don't expect many users to understand what OpenAI or ChatGPT is. They'll think of it all as Apple's product because it's integrated into the OS and the Apps, and it came with your phone. Literally, if you write Apps for a living, you hear people telling you that your App should be free because the phone cost so much, and why should anyone have to pay for apps on top of that cost? Aren't you being paid by Apple to make the App for the phone? You're just greedy and double-dipping.

Yes there will be an alert box telling them. And you know what, people don't read alert boxes, even if you tell them to so that you can help fix something. To quote a regular conversation I have with my relatives: "It made an error. Can you fix it?" "What error?" "I don't know, I didn't read it, it was too long and didn't make any sense". "But I've told you I can't fix things if I don't know what is wrong!" "How am I supposed to remember all that gobbledygook?"

FWIW, Mira Murati says that OpenAI's stuff in the labs isn't much more advanced than what users get to use.

OpenAI just appointed an retired US army general, who also ran the NSA and us cyber command until February this year to its board. Why would someone like that end up on the board? My guess: to sell services to the spooks, military & cyber-defense people.

It's somewhat ironic. Geoffrey Hinton once said in an interview that he was counting on his student Ilya Sutskever to ensure advanced AI would not end up being used by the military, but then Ilya left/was pushed out of OpenAI. Geoffrey believed so strongly in his anti-war position that he moved from the US to Canada to avoid supporting the US military adventure in Vietnam by paying his taxes, IIRC. So yeah. He must be happy.