Apple Intelligence News Notification Summaries
Apple Intelligence, launched in the UK earlier this week, uses artificial intelligence (AI) to summarise and group together notifications.
This week, the AI-powered summary falsely made it appear BBC News had published an article claiming Luigi Mangione, the man arrested following the murder of healthcare insurance CEO Brian Thompson in New York, had shot himself. He has not.
A news summary from Apple falsely claimed darts player Luke Littler had won the PDC World Championship - before he even played in the final.
The incorrect summary was written by artificial intelligence (AI) and is based on a BBC story about Littler winning the tournament semi-final on Thursday night.
Within hours on Friday, another AI notification summary falsely told some BBC Sport app users that Tennis great Rafael Nadal had come out as gay.
The ads for Apple Intelligence have mostly been noted for what they show, but there is also something missing: in the fine print and in its operating systems, Apple still calls it a “beta” release, but not in its ads. Given the exuberance with which Apple is marketing these features, that label seems less like a way to inform users the software is unpolished, and more like an excuse for why it does not work as well as one might expect of a headlining feature from the world’s most valuable company.
[…]
Apple has also, rarely, applied the “beta” label to features in regular releases which are distributed to all users, not just those who signed up. This type of “beta” seems less honest. Instead of communicating this feature is a work in progress, it seems to say we are releasing this before it is done. Maybe that is a subtle distinction, but it is there. One type of beta is testing; the other type asks users to disregard their expectations of polish, quality, and functionality so that a feature can be pushed out earlier than it should.
[…]
This all seems like a convoluted way to evade full responsibility of the Apple Intelligence experience which, so far, has been middling for me. Genmoji is kind of fun, but Notification Summaries are routinely wrong. Priority messages in Mail is helpful when it correctly surfaces an important email, and annoying when it highlights spam. My favourite feature — in theory — is the Reduce Interruptions Focus mode, which is supposed to only show notifications when they are urgent or important. It is the kind of thing I have been begging for to deal with the overburdened notifications system. But, while it works pretty well sometimes, it is not dependable enough to rely on.
I don’t think that the vast majority of people know what beta means. Apple has been promoting the shit out of these features, and putting beta in a footnote.
Xe Iaso (via Hacker News):
This phrases a literal scam message in ways that make me think immediate action is required. You can see how this doesn’t scale, right?
[…]
Even more, if you have Apple Intelligence enabled for some of the other features but disable notification summaries because you find them worthless, you can get your notifications delayed up to five seconds. It’s kind of depressing that telling your computer to do less work makes the result take longer than doing more work.
Additionally, none of the summarization features work on my iPhone and I can’t be bothered to figure out why and fix it. I personally don’t find them useful. I just leave them enabled on my MacBook so that notification delivery is not impacted.
[The] whole vibe of Apple Intelligence is off-putting and feels like a not-ready-for-primetime suite of features that make the user experience worse.
Apple is working on an update for Apple Intelligence that will cut down on confusion caused by inaccurate summaries of news headlines, Apple told BBC News. In a statement, Apple said software coming soon will clarify when notifications have been summarized by Apple Intelligence.
[…]
There have been several prior events where Apple Intelligence provided incorrect details from incoming news app notifications. In November, Apple Intelligence suggested Israeli Prime Minister Benjamin Netanyahu had been arrested, incorrectly interpreting a story from The New York Times.
[…]
Apple Intelligence notification summaries are an opt-in feature and they can be disabled.
My understanding is that they are opt-out in that once you opt into Apple Intelligence in general, you have to opt out of the notification summaries if you don’t want them. And, crucially, this is at the user level. There is no way for an app developer such as the BBC to prevent its app’s notifications from being summarized.
Apple is promoting the hell out of Apple Intelligence to consumers, and its advertisements hide, rather than emphasize, its “beta” quality.
The promotion of a feature is an implicit encouragement to, you know, actually use it.
[…]
Apple Intelligence notification summaries are marked with an icon/glyph, sort of like the “↪︎” Unicode glyph with a few horizontal lines to suggest text encapsulated by the arrow — a clever icon to convey an abstract concept, to be sure.
The meaning of that icon/glyph is not at all obvious unless you know to look for it, and most users — even those who opted in to Apple Intelligence understanding that it was “beta” and might produce erroneous results — don’t know to look for that particular glyph.
[…]
I can also see why Apple doesn’t want to offer such an option to developers. To whom do notifications belong — the developer of the app that generates them, or the user who is receiving them?
The statement uses the beta tag it has placed on Apple Intelligence features as a shield, while promising to add a warning label to AI-generated summaries in the future. It’s hard to accept “it’s in beta” as an excuse when the features have shipped in non-beta software releases that are heavily marketed to the public as selling points of Apple’s latest hardware. Adding a warning label also does not change the fact that Apple has released a feature that at its core consumes information and replaces it with misinformation at a troubling rate.
Apple is shipping these AI-based features rapidly, and marketing them heavily, because it fears that its competitors so far out in front that it’s a potentially existential issue. But several of these features simply aren’t up to Apple’s quality standards, and I worry that we’ve all become so inured to AI hallucinations and screw-ups that we’re willing to accept them.
[…]
So what can Apple do now? A non-apology and the promise of a warning label isn’t enough. The company should either give all apps the option of opting out of AI summaries, or offer an opt-out to the developers of specific classes of apps (like news apps). Next, it should probably build separate pathways for notifications of related content (a bunch of emails or chat messages in a thread) versus unrelated content (BBC headlines, podcast episode descriptions) and change how the unrelated content is summarized.
I side with Apple in not giving developers the option to opt out of notification summaries, and (b) that I’m a bit more of the mind that Apple can address this by somehow making it more clear which notifications are AI-generated summaries. Like, perhaps instead of their “↪︎” glyph, they could use the 🤪 emoji.
If Apple Intelligence summarizes your notifications then Apple should badge it with their Apple logo. Not some weird cog or brain or some other such icon. Put your name on it! Apple is the one presenting this information to you and they should be held accountable for the veracity of it. Put your highly regarded Apple logo on your AI work or get outta here. It’s either an Apple product or it’s not.
The problem with Apple’s approach is that it’s summarizing a headline, which is itself a summary of an article written by a human being. As someone who has written and rewritten thousands of headlines, I can reveal that human headline writers are flawed, some headlines are just not very good, and that external forces can lead to very bad headlines becoming the standard.
Specifically, clickbait headlines are very bad, and an entire generation of headline writers has been trained to generate teaser headlines that purposefully withhold information in order to get that click.
[…]
Summarizing summaries isn’t working out for Apple, but more broadly I think there’s something to the idea of presenting AI-written headlines and summaries in order to provide utility to the user. As having an LLM running all the time on our devices becomes commonplace, I would love to see RSS readers (for example) that are capable of rewriting bad headlines and creating solid summaries. The key—as Artifact learned—is to build guardrails and always make it clear that the content is being generated by an LLM, not a human.
Starting to think Apple might regret sticking its name in front of ‘Intelligence’ for all its AI stuff. Notifications are a disaster. Image Email categories are a disaster. And so on. Then again, the ad campaign is somehow even worse than all of that.
The sad thing is, there are good elements to Apple AI/ML. Prompt-based memories in the Photos app. Auto-tagging. Accessibility features like Personal Voice. But so much attention has been grabbed by flashy stuff that did not – and in some cases could not – work.
The Apple Intelligence vs BBC story is a microcosm of the developer story for the feature. We’re soon expected to vend up all the actions and intents in our apps to Siri, with no knowledge of the context (or accuracy) in which it will be presented to the user. Apple gets to launder the features and content of your apps and wrap it up in their UI as ‘Siri’ — that’s the developer proposition Apple has presented us. They get to market it as Apple Intelligence, you get the blame if it goes awry.
Apple plans to scale up its News app by adding new countries to the platform beyond the US, Canada, the UK, and Australia, according to the Financial Times.
The plans reportedly include building its locally focused news coverage in the UK, as well as bringing its puzzles section to the country which is currently limited to the US and Canada.
With Apple News, Apple does have access to the full article text. Maybe it will use this to dogfood a way of making this available for notification summaries.
Previously:
- Misguided Apple Intelligence Ads
- Apple Intelligence in macOS 15.2 and iOS 18.2
- Apple Intelligence in macOS 15.1 and iOS 18.1
- Beta for Apple Intelligence in Apple Mail
Update (2025-01-10): Nick Heer:
Apple should not be putting its name or logo on something it does not stand behind, and it should stand behind everything it ships. It supposedly cannot “ship junk”, but it is obviously not yet proud of the way these notifications were summarized — it is making changes, after all. But will it be courageous enough to attach its valuable brand to the output of its own large language model? I would bet against it, but it should.
See also: John Gruber.
Cam Wilson (via Hacker News):
Screenshots from iPhone users show this new suite of AI-powered features appears unable to distinguish between messages sent by real individuals and organisations and fake requests made by scammers imitating others. In fact, the AI-powered features may even make it harder for users to initially distinguish between real and fake messages.
Steve, a pseudonym granted as his work has not authorised him to speak to the media, was surprised to see that his recently updated iPhone had prioritised and summarised an email saying that he had to lodge a income statement to the Australian Tax Office.
Update (2025-01-13): Chris Pepper:
The bad scenario is users not noticing “an unexpected notification summary” — in which case people will be misinformed, and will misattribute any misinformation to the BBC. Users who don’t notice the discrepancy won’t report anything to Apple. So Apple is describing the wrong problem with an inapplicable solution.
[…]
The expectation that someone will read an (incorrect) AI summary, tap/click to read the original news piece, and of course notice the discrepancy, devalues AI completely. If news notifications are just links to articles, there’s no need to summarize them at all. The points of a notification are a. to give you the essential information, and b. to provide more detail if interested.
[…]
This is not about unexpected summaries. It’s about incorrect notifications. Every news summary is expected, except when you already know something big just happened, and wait for your phone to catch up and tell you about it. ‘Unexpected’ is a weasel word from a PR person or lawyer who says Apple cannot admit that this is all about falsehood/misinformation/incorrect notifications.
Apple is solving their problem—of being liable for misattributing misinformation. But this does nothing for the customer’s problem—that the notification is incorrect—except insomuch as it hints that this is a feature that could be turned off.
Because I read quickly, I see no reason to ask Apple Intelligence to generate a summary of a Web page or a conversation in Mail. The downside of losing detail and nuance—and of possible errors—outweighs the upside of saving a few minutes of reading time. Notification summaries are even worse; for me, they save seconds at most and often introduce confusion by summarizing unrelated news articles or information that has changed multiple times within the summary period. The main utility I see for notification summaries is to reduce the irritation of too many notifications from chatty conversations or overactive apps, but Apple has already addressed that by grouping notifications.
While AI-generated summaries raise valid concerns, it’s essential to recognize that human-created summaries permeate nearly everything we read. For instance, every email message and discussion forum post has a subject line that’s supposed to summarize the message’s intent. People often write poor subject lines, but they remain an essential form of summary—one that AI could actually help improve.
[…]
To summarize—I had to!—summaries offer a different value proposition for everyone. Reading speed, language fluency, topical understanding, display space, and other factors play into how valuable a summary of a particular length will be in any given situation. You should ask for AI-generated summaries only when they will provide actual value and you can verify their accuracy when it matters. Finally, remember that just because something can be summarized doesn’t mean it should be.
What gets me about the Mail summaries, besides having to scroll and click to see them, is that they are so slow to generate. By the time the summary is ready I could have already skimmed the e-mail myself. Am I supposed to start doing this while waiting? Then I would be essentially reading the e-mail twice. Or zone out for a bit? I certainly don’t want to context switch while I wait. I don’t understand why the summaries aren’t pre-generated for new messages or ones that are next in the list for me to read.
7 Comments RSS · Twitter · Mastodon
Once in a while I get it right: https://mjtsai.com/blog/2024/06/10/apple-intelligence-announced/#comment-4104609
I don't get what problem AI concocted notification summaries are solving.
Hell of a risk to their reputation Apple have knowingly taken for little or no value.
The problem Apple tried to solve was that more and more people were saying "Apple are late with AI. They are missing the boat. They are old and slow"
So they clearly had a bunch of workshops where mostly marketing people were tasked with integrating "ai" into iOS in as many ways as possible. Few, if any, developers had a say.
Similar to when people were laughing at their skeuomorphic design language and they rushed their new flat look out the door.
I wish Apple had held fast and said "When there is something that will actually benefit our users we will launch it." But 'the market' needed action!
And then there's that stupid fucking name they had to put on it.
There's so much that's right about your comment. I have rarely read remarks on how really ffff dumb 'apple intelligence' is as a name. The Apple presentations are already really weird, weird people, twisting language, weird tone of delivery - seeing 'apple intelligence' up on the screen was hilarious.
https://mjtsai.com/blog/2025/01/07/apple-intelligence-news-notification-summaries/#comment-4219919
Yeah I get why they are rushing out AI features but why summarise headlines?
Headlines are already summarised. There's nothing to be gained and a high chance of losing trust.
One of many categories that should be left untouched by the random word-salad generators.
I think the sad truth is that there aren't that many every day use cases where "ai" would be beneficial. So Apple chose to cram in a bunch of useless crap. Much like when Google added a generate a background image in android two years ago.
It's desperation.
If they ever make "ai" dependable enough to actually do the boring shit like trawling through all emails and put together a nice summary of hotel and flight bookings for a trip, or add reminder for the ever changing activities I need to take the kids to then we're talking.
I'd *love* it if the OS could scan my incoming emails from my sons diving class and update the family calendar with changed times, cancelled lessons, upcoming competitions and etc.
But I'm not holding my breath. Instead we get Apple generating fake news and calling t intelligent.
Screwing with news headlines specifically is a bad idea when mistrust of journalists is already high. If the AI summaries make it harder to tell what’s true and what’s not, the beneficiaries will be authoritarian governments, not Apple’s users. I wrote a post about this here: https://esham.io/2025/01/apple-ai-news
