Archive for October 24, 2024

Thursday, October 24, 2024

Apple Intelligence in macOS 15.2 and iOS 18.2

Six Colors:

On Wednesday, Apple rolled out developer betas of iOS 18.2, iPadOS 18.2, and macOS 15.2, which run Apple Intelligence features previously seen only in Apple’s own marketing materials and product announcements: Three different kinds of image generation, ChatGPT support, Visual Intelligence, expanded English language support, and Writing Tools prompts.

[…]

It’s still English-only for now, but English speakers in Canada, the United Kingdom, Australia, New Zealand, and South Africa will be able to use Apple Intelligence in their versions of English.

Juli Clover:

Apple introduced an updated version of the Mail app with built-in categorization.

I still don’t see the categories feature in the Mac version of Mail, although there are some invisible menu commands that show up when searching with the Help menu.

From a SpamSieve perspective, I note that Mail’s data store is still at version 10 but that the schema has changed from both macOS 14 and from macOS 15.0 (in seemingly backwards compatible ways).

John Gruber:

These developer betas also contain new APIs for third-party apps: the Writing Tools API (which will allow any text app to support the features only Apple’s first-party apps have access to in iOS 18.1 and MacOS 15.1), Genmoji API (so third-party messaging apps can support them like Messages will), and Image Playground API.

Howard Oakley:

Apple reassured us that “if you’re using any of the standard UI frameworks to render text fields, your app will automatically get the ability to use Writing Tools.” But that appears to make the assumption that the text view is already using TextKit 2, and the only documentation that I can find about that states that NSTextViews need to be opted into that with additional code. However, the class documentation for NSTextView doesn’t even mention TextKit 2, although it does now include some information about support for Writing Tools.

See also: Marcin Krzyzanowski.

M.G. Siegler:

One more thing: also baked into iOS 18.2 is the ability to set default apps for Mail, Browser, Messages, etc. This isn't just for EU users, but for everyone.

Previously:

Update (2024-10-29): John Gruber:

The image generation features (Image Playground, Genmoji, Image Wand) in the next round of Apple Intelligence, in the beta releases of iOS 18.2 and MacOS 15.2 that dropped last week, require a separate waiting list. I signed up for that a few hours after the betas were released last Wednesday, October 23, and I’m still waiting as I type this. The only people I know who have access to the image generation features are those who signed up for it within the first hour — maybe less — of the betas appearing.

Update (2024-11-05): Tyler Fox:

New in iOS 18.2 developer beta 2! Apps can integrate with Siri and Apple Intelligence so users can ask questions about onscreen content, such as photos and documents. You can adopt the API from SwiftUI, UIKit, and/or AppKit — see details & sample code.

Update (2024-11-08): Juli Clover:

The version of Image Playground available in the iOS 18.2, iPadOS 18.2, and macOS Sequoia 15.2 betas only offers animation and illustration as style options, leaving us wondering if sketch might be added a later time.

It looks like the answer might be no, as Apple has removed Sketch from the Image Playground app description.

Update (2024-12-13): Apple:

Apple today announced the release of iOS 18.2, iPadOS 18.2, and macOS Sequoia 15.2, introducing a brand-new set of Apple Intelligence features that will elevate users’ experience with iPhone, iPad, and Mac, and builds on the first set of capabilities already introduced.

[…]

Additional Apple Intelligence capabilities will be available in the months to come. Siri will be even more capable, with the ability to draw on a user’s personal context to deliver intelligence that’s tailored to them. Siri will also gain onscreen awareness, and will be able to take hundreds of new actions in and across Apple and third-party apps. Priority Notifications will also surface what’s most important. In addition, users will be able to create images in Image Playground in a Sketch style, an academic and highly detailed style that uses a vibrant color palette combined with technical lines to produce realistic drawings.

Ryan Christoffel:

Layered Recordings is that Voice Memos feature. It’s exclusive to the iPhone 16 Pro and 16 Pro Max, and available now in iOS 18.2.

Tim Hardwick:

Categories is the default view after updating to the new software. Fortunately, Apple makes it simple to switch back to the traditional list view.

Stephen Hackett:

The splashiest feature is Image Playgrounds, which allows users to create images based on photos in their image libraries[…]

[…]

I’m not that impressed with what Apple has done here [with Genmoji], but I think that’s okay.

I am glad Apple is not creating lifelike images with these tools, but of course, many services are doing that. This week, OpenAI launched SORA, its video creation tools.

Six Colors:

These features are, on the whole, more ambitious than the initial batch released back in in October, and some of them build on those features: for example, the ability to now generate specific changes to text in Writing Tools. This also marks the first third-party integration of generative AI features into Apple’s own platforms, with the ability to connect to ChatGPT.

Apple Intelligence features are also expanding geographically with these releases, coming to more versions of English[…]

Jason Snell:

One of my complaints is that you can’t just make generic figures of people—you have to choose actual people in your library.

This is wrong. You can do it—I just completely missed the feature, because it wasn’t positioned or labeled in a way that made me understand what I was looking at.

Juli Clover:

Apple today updated Pages, Numbers, and Keynote for Mac and iOS with support for Apple Intelligence features like ChatGPT Siri integration, Writing Tools, and Image Playground that were largely introduced in iOS 18.2, iPadOS 18.2, and macOS Sequoia 15.2.

Juli Clover:

Below, we’ve highlighted the Apple Intelligence features we know are still in development and that are slated for iOS 18.3 and iOS 18.4, updates coming in 2025.

Previously:

Update (2024-12-16): Jeff Carlson (Mastodon):

But what if you don’t want Apple Intelligence foisted on you?

[…]

It’s possible to turn off Apple Intelligence entirely, or to disable it for specific apps. And if you decide you want to jump back into the AI stream, you can easily turn it back on. Here’s what you need to know.

Pierre Igot:

The first thing I did after updating my iPhone to iOS 18.2 was of course to turn categorizing off in Mail, which had to be done SEVERAL times, once for each inbox. And then I watched as Mail continued to display “Categorizing…” at the bottom of the screen, AFTER I had turned categorizing off.

Mario Guzmán:

The fact that you have to swipe on the four main category buttons to see that a 5th button exists is awful UX.

You shouldn’t rely on users experimenting and trying various gestures to see what and what doesn’t exist in your UI.

Ben Lovejoy (Hacker News):

A new survey suggests that Apple Intelligence matters to iPhone buyers, but the majority say that the initial features add little to no value. It remains to be seen whether Genmoji and ChatGPT integration will change that view.

Things are even worse for Samsung smartphones, with an even greater majority of owners saying they can’t see much point in the AI features offered …

Mario Guzmán:

I came out to see my bestie from college and her husband. We all did Computer Science at Cal Poly.

Anyway, neither of them had updated to iOS 18.1 even and weren’t aware of Apple Intelligence. They both have the iPhone 16 Pro.

They only upgraded while I was here bc they liked the new Siri edge to edge animation/UI.

Markus Müller-Simhofer:

If you are using macOS 15.2, please be careful with those priority alerts. It listed a fraud email as priority for me. 😕

Update (2024-12-18): Joe Rosensteel:

People have asked for filters/rules for years for Mail on iOS, and Apple didn’t give them to us… until, all of a sudden, we’ve got a few hard-coded invisible rules that users can nudge a little. We can’t be trusted with Smart Mailboxes or labels, but we do have three immutable categories that all email is supposed to fit into.

[…]

Apple skipped the opt-in step, which I suspect will engender far more ire than had they gone the traditional route. There will always be people who are resistant to any change, but this release strategy isn’t helping. What’s worse, there’s no universal “go back” switch people can flip. Some of the feature toggles are in Settings, and some are in hidden menu buttons inside the Mail app.

Lee Bennett:

My macOS Mail.app smart mailboxes sync to other desktop devices. Why can’t I have them sync to iOS as well? That would be FAR more useful to me than this categorization swing and miss.

Previously:

Update (2024-12-19): Adam Chandler:

I’d rate Mobile Mail on iOS as a dumpster fire and I’m quickly considering replacing the Mail icon with Outlook. That’s how bad it’s gotten.

I disabled categories and the AI features but search is continually broken pushing me to “connect to wifi and a charger to search your mail” When I click a new mail notification, it takes 10 seconds for the message to load.

Joe Rosensteel (Mastodon):

The post on Six Colors is a much more focused, and more relatable, blog post that went right into the problems with Categories. That’s why Jason’s a great editor, folks. I’ll include the less interesting parts here as a “bonus” for people that like to read about my frustrations.

J.P. Wing:

Ok folks, since iOS’s native mail app has gone all sideways with Apple Intelligence, recommendations for an alternative on the iPhone?

Joshua Nozzi:

Mine randomly decides to not inform me of new mail since I turned AI off. ShIpIt!!!

Update (2024-12-23): Dr. Drang:

I thought that changing from Categories View to List View would be enough to let AI know that I wanted all of my unread messages to be counted in the badge. But no. I still had to open the Settings app and work my way through Apps>Mail>Notifications>Customize Notifications until I reached the screen that let me switch the badge to show the count of all unread messages.

Update (2025-01-07): Jesse Squires:

The Mail.app in iOS 18.2 stopping showing notification badges for me, even though I had them turned on.

I also turned off all the new “category” nonsense.

Turns out, there’s a second (very hidden) notification setting dealing with “categories” that you also have to fix.

Update (2025-01-09): Cabel Sasser:

i’m embarrassed to ask this, but what does this new yellow Mail swipe action do? Is it “Mute”?

I looked in Prefs to try to teach myself, and it’s not even listed as an option!?

Craig Hockenberry:

The whole Mail categorization feature confused me completely. And it was hard to figure out how to disable it.

Amazing that this was enabled by default for millions of folks.

Update (2025-01-10): Craig Hockenberry:

Why is “Writing Tools” the first thing shown in Safari’s context menu? It’s pointless and makes it harder to get at the things I actually use: “Lookup” and “Translate”.

Some manager at Apple decided that showing off a new feature was more important than usability.

I’m also seeing issues where the orange writing tools icon shows up, unwanted, next to some text and the only way to get rid of it is to quit and relaunch the app.

Apple Intelligence in macOS 15.1 and iOS 18.1

Joanna Stern:

Apple will launch iOS 18.1 next week, bringing its much anticipated generative-AI tools to the iPhone 15 Pro models and the new iPhone 16 lineup. It will be available for most newer iPads and Macs, too.

If you’re expecting AI fireworks, prepare for AI…sparklers. Back in June, at the company’s annual developers conference, executives showed off do-it-yourself emojis, ChatGPT integration and a Siri that can recall the name of a person you met months ago. Apple has even been running ads for some features. None are in this release.

[…]

I’ve been testing Apple Intelligence on my iPhone and iPad. Apple’s ability to build tools right into the operating systems is undeniably powerful and convenient. But many are half-baked. I asked Federighi to explain the features—and Apple’s broader AI strategy.

John Gruber:

It’s a very good interview, and also available on YouTube.

[…]

But as Stern herself points out in the article, the features that are shipping are genuinely useful. Notification summaries are good — the occasional mistakes can be funny, but overall it’s solid, and especially helpful for batches of notification from the same app or group text. The Clean Up unwanted-object-remover in Photos is great.

M.G. Siegler:

The first version of Apple Intelligence, which has been in beta testing for a few months now and is rolling out broadly next week, is pretty underwhelming. There’s just not much there. Not a lot beyond perhaps notification summaries that you’re going to be using all the time.

Tim Hardwick:

The most significant Siri enhancements are scheduled for iOS 18.4 around March 2025. These include onscreen awareness for contextual commands, personal context for better understanding of user data, and expanded app control capabilities. Initially, Apple Intelligence will only support U.S. English, with additional languages planned for next year.

Om Malik:

If my memory serves me correctly, it’s roughly the same number Apple shared during the WWDC keynote. So, essentially flat. While 1.5 billion might appear big, when it comes to internet scale, it isn’t such a large number for a company the size of Apple. I looked up the number of active Apple devices. That number is estimated to be 2.2 billion devices — I assume this includes phones, computers, watches, headphones, TV-streaming devices, and speakers. So 1.5 billion requests a day is actually far less than one daily request per active device.

Howard Oakley:

Writing Tools don’t themselves generate new content in text, but use the original text to produce derivatives. I’m particularly looking forward to using its proofreading feature, which can suggest improvements that I can choose to ignore, or adapt to my own style, as I wish.

Adam Engst:

I’ve relied on Grammarly for years for proofreading. It catches typos, doubled words, and extra spaces, and its newer AI-powered features sometimes make helpful suggestions for recasting awkward sentences. I’m slightly annoyed that Grammarly’s proofreading tools are so helpful, but it’s challenging to edit your own text to a professional level, and Grammarly can identify errors much faster than I can. Don’t assume that tools like Apple Intelligence’s proofreading capabilities for helping with grammar, word choice, and sentence structure are necessarily a crutch. They may be for some people, but even people who care about their writing can still benefit from some suggestions while ignoring unhelpful ones.

Ben Lovejoy:

AI suggested a total of six changes to my piece, of which three were duplicates – adding periods to bullet-point text.

[…]

Overall, though, a really excellent job.

John Gruber:

I am very likely underselling how valuable the new writing tools might prove to people trying to write in a second language, or who simply aren’t capable of expressing themselves well in their first language.

Juli Clover:

This guide goes over everything you can do with Writing Tools, where you can use them, and what you need to access the feature.

Previously:

Update (2024-10-28): Apple (Hacker News):

Apple today announced the first set of Apple Intelligence features for iPhone, iPad, and Mac users is now available through a free software update with the release of iOS 18.1, iPadOS 18.1, and macOS Sequoia 15.1. Apple Intelligence is the personal intelligence system that harnesses the power of Apple silicon to understand and create language and images, take action across apps, and draw from personal context to simplify and accelerate everyday tasks while taking an extraordinary step forward for privacy in AI. Today marks the availability of the first set of features, with many more rolling out in the coming months.

Six Colors:

It’s unquestionable that Apple is putting its weight behind these efforts, but what’s been less clear is just how effective and useful these tools will be. Perhaps unsurprisingly, for anybody who has used similar generative AI tools, the answer is a definite maybe.

[…]

Despite Apple’s marketing of a new and improved Siri, the voice assistant hasn’t changed that much with this first set of Apple Intelligence capabilities. The most obvious “new feature” is actually a new look: on iOS and iPadOS, instead of the little glowing orb that used to indicate Siri had been activated, you’ll now see a colorful wash over the entire screen, accompanied by a “ripple” effect.

Previously:

Update (2024-10-30): Kirk McElhearn:

These writing tools could be useful for non-native speakers or people with limited writing skills. But for any serious writing, they are limited and problematic. Someone in a hurry may accept rewrites without checking and later discover that their text has been corrupted, their style flattened, and their message obfuscated.

iA:

Apple’s Intelligence misses a crucial step in the process. Writing Tools will simply replace your original text. If you can’t see the edits. The more you use it, the more you risk losing control over what you wrote.

Update (2024-10-31): Joe Rosensteel:

  1. Clean Up (YIKES)
  2. Pixelmator (yuck)
  3. Retouch (ok, and what I used originally)
  4. Lightroom (many removal options that all seem viable)

Update (2024-12-13): Chance Miller:

The BBC isn’t pleased with Apple Intelligence’s notification summary feature. The corporation says that the notification summary feature “generated a false headline” about Lugi Magione, who was arrested this week as the suspected killer of the United HealthGroup CEO.

The notification summary in question suggested that Mangione had shot himself: “Luigi Mangione shoots himself; Syrian mother hopes Assad pays the price; South Korea police raid Yoon Suk Yeol’s office.”

Update (2024-12-16): Mario Guzmán:

I think iOS Notification Summaries should have an easier way to be disabled and shouldn’t even be allowed for any app that delivers the news to users.

Is there a way for developers to even opt out altogether from notification summaries?

Benjamin Mayo:

what a UI

Postbox Discontinued

Adam Engst (Hacker News, MacRumors Forum):

Sad news. The longstanding email client Postbox has been acquired and shut down by eM Client, described by the announcement as “a leading email platform for Windows and macOS that combines email, calendars, tasks, contacts, notes, and chat into a single, easy-to-use application.”

[…]

On the other side of the equation, I’ve never heard of any Mac users relying on eM Client, despite its cross-platform status. I was going to say that I’d never heard of it at all, but searching my email reveals that I tested it briefly in February 2022. I suspect it’s a Windows app that has been clumsily ported to macOS. (Ironically, I was just encouraged by a PR person to look at Mailbird, another Windows email client that just released a version that runs on the Mac but bears little resemblance to a true Mac app.) eM Client gained iOS and Android clients only this year.

There’s a migration guide. I know that eM Client has some dedicated fans, and I’ve received requests over the years for SpamSieve to work with it. Unfortunately, eM Client does not support plug-ins or have any meaningful AppleScript support. Some customers are running Apple Mail in the background to filter their mail with SpamSieve. They can correct any mistakes from within eM Client by moving messages to special TrainSpam and TrainGood mailboxes. I’m open to working with eM Client if they would like to add direct integration.

Ric Ford:

Postbox was based on Mozilla’s cross-platform, open-source code Thunderbird email app, which does provide native Apple Silicon code, along with timely development and updates, security fixes, and additional features, including calendars.

Copyright © 2000–2025 Michael Tsai.