Apple Intelligence News Notification Summaries
Apple Intelligence, launched in the UK earlier this week, uses artificial intelligence (AI) to summarise and group together notifications.
This week, the AI-powered summary falsely made it appear BBC News had published an article claiming Luigi Mangione, the man arrested following the murder of healthcare insurance CEO Brian Thompson in New York, had shot himself. He has not.
A news summary from Apple falsely claimed darts player Luke Littler had won the PDC World Championship - before he even played in the final.
The incorrect summary was written by artificial intelligence (AI) and is based on a BBC story about Littler winning the tournament semi-final on Thursday night.
Within hours on Friday, another AI notification summary falsely told some BBC Sport app users that Tennis great Rafael Nadal had come out as gay.
The ads for Apple Intelligence have mostly been noted for what they show, but there is also something missing: in the fine print and in its operating systems, Apple still calls it a “beta” release, but not in its ads. Given the exuberance with which Apple is marketing these features, that label seems less like a way to inform users the software is unpolished, and more like an excuse for why it does not work as well as one might expect of a headlining feature from the world’s most valuable company.
[…]
Apple has also, rarely, applied the “beta” label to features in regular releases which are distributed to all users, not just those who signed up. This type of “beta” seems less honest. Instead of communicating this feature is a work in progress, it seems to say we are releasing this before it is done. Maybe that is a subtle distinction, but it is there. One type of beta is testing; the other type asks users to disregard their expectations of polish, quality, and functionality so that a feature can be pushed out earlier than it should.
[…]
This all seems like a convoluted way to evade full responsibility of the Apple Intelligence experience which, so far, has been middling for me. Genmoji is kind of fun, but Notification Summaries are routinely wrong. Priority messages in Mail is helpful when it correctly surfaces an important email, and annoying when it highlights spam. My favourite feature — in theory — is the Reduce Interruptions Focus mode, which is supposed to only show notifications when they are urgent or important. It is the kind of thing I have been begging for to deal with the overburdened notifications system. But, while it works pretty well sometimes, it is not dependable enough to rely on.
I don’t think that the vast majority of people know what beta means. Apple has been promoting the shit out of these features, and putting beta in a footnote.
Xe Iaso (via Hacker News):
This phrases a literal scam message in ways that make me think immediate action is required. You can see how this doesn’t scale, right?
[…]
Even more, if you have Apple Intelligence enabled for some of the other features but disable notification summaries because you find them worthless, you can get your notifications delayed up to five seconds. It’s kind of depressing that telling your computer to do less work makes the result take longer than doing more work.
Additionally, none of the summarization features work on my iPhone and I can’t be bothered to figure out why and fix it. I personally don’t find them useful. I just leave them enabled on my MacBook so that notification delivery is not impacted.
[The] whole vibe of Apple Intelligence is off-putting and feels like a not-ready-for-primetime suite of features that make the user experience worse.
Apple is working on an update for Apple Intelligence that will cut down on confusion caused by inaccurate summaries of news headlines, Apple told BBC News. In a statement, Apple said software coming soon will clarify when notifications have been summarized by Apple Intelligence.
[…]
There have been several prior events where Apple Intelligence provided incorrect details from incoming news app notifications. In November, Apple Intelligence suggested Israeli Prime Minister Benjamin Netanyahu had been arrested, incorrectly interpreting a story from The New York Times.
[…]
Apple Intelligence notification summaries are an opt-in feature and they can be disabled.
My understanding is that they are opt-out in that once you opt into Apple Intelligence in general, you have to opt out of the notification summaries if you don’t want them. And, crucially, this is at the user level. There is no way for an app developer such as the BBC to prevent its app’s notifications from being summarized.
Apple is promoting the hell out of Apple Intelligence to consumers, and its advertisements hide, rather than emphasize, its “beta” quality.
The promotion of a feature is an implicit encouragement to, you know, actually use it.
[…]
Apple Intelligence notification summaries are marked with an icon/glyph, sort of like the “↪︎” Unicode glyph with a few horizontal lines to suggest text encapsulated by the arrow — a clever icon to convey an abstract concept, to be sure.
The meaning of that icon/glyph is not at all obvious unless you know to look for it, and most users — even those who opted in to Apple Intelligence understanding that it was “beta” and might produce erroneous results — don’t know to look for that particular glyph.
[…]
I can also see why Apple doesn’t want to offer such an option to developers. To whom do notifications belong — the developer of the app that generates them, or the user who is receiving them?
The statement uses the beta tag it has placed on Apple Intelligence features as a shield, while promising to add a warning label to AI-generated summaries in the future. It’s hard to accept “it’s in beta” as an excuse when the features have shipped in non-beta software releases that are heavily marketed to the public as selling points of Apple’s latest hardware. Adding a warning label also does not change the fact that Apple has released a feature that at its core consumes information and replaces it with misinformation at a troubling rate.
Apple is shipping these AI-based features rapidly, and marketing them heavily, because it fears that its competitors so far out in front that it’s a potentially existential issue. But several of these features simply aren’t up to Apple’s quality standards, and I worry that we’ve all become so inured to AI hallucinations and screw-ups that we’re willing to accept them.
[…]
So what can Apple do now? A non-apology and the promise of a warning label isn’t enough. The company should either give all apps the option of opting out of AI summaries, or offer an opt-out to the developers of specific classes of apps (like news apps). Next, it should probably build separate pathways for notifications of related content (a bunch of emails or chat messages in a thread) versus unrelated content (BBC headlines, podcast episode descriptions) and change how the unrelated content is summarized.
I side with Apple in not giving developers the option to opt out of notification summaries, and (b) that I’m a bit more of the mind that Apple can address this by somehow making it more clear which notifications are AI-generated summaries. Like, perhaps instead of their “↪︎” glyph, they could use the 🤪 emoji.
If Apple Intelligence summarizes your notifications then Apple should badge it with their Apple logo. Not some weird cog or brain or some other such icon. Put your name on it! Apple is the one presenting this information to you and they should be held accountable for the veracity of it. Put your highly regarded Apple logo on your AI work or get outta here. It’s either an Apple product or it’s not.
The problem with Apple’s approach is that it’s summarizing a headline, which is itself a summary of an article written by a human being. As someone who has written and rewritten thousands of headlines, I can reveal that human headline writers are flawed, some headlines are just not very good, and that external forces can lead to very bad headlines becoming the standard.
Specifically, clickbait headlines are very bad, and an entire generation of headline writers has been trained to generate teaser headlines that purposefully withhold information in order to get that click.
[…]
Summarizing summaries isn’t working out for Apple, but more broadly I think there’s something to the idea of presenting AI-written headlines and summaries in order to provide utility to the user. As having an LLM running all the time on our devices becomes commonplace, I would love to see RSS readers (for example) that are capable of rewriting bad headlines and creating solid summaries. The key—as Artifact learned—is to build guardrails and always make it clear that the content is being generated by an LLM, not a human.
Starting to think Apple might regret sticking its name in front of ‘Intelligence’ for all its AI stuff. Notifications are a disaster. Image Email categories are a disaster. And so on. Then again, the ad campaign is somehow even worse than all of that.
The sad thing is, there are good elements to Apple AI/ML. Prompt-based memories in the Photos app. Auto-tagging. Accessibility features like Personal Voice. But so much attention has been grabbed by flashy stuff that did not – and in some cases could not – work.
The Apple Intelligence vs BBC story is a microcosm of the developer story for the feature. We’re soon expected to vend up all the actions and intents in our apps to Siri, with no knowledge of the context (or accuracy) in which it will be presented to the user. Apple gets to launder the features and content of your apps and wrap it up in their UI as ‘Siri’ — that’s the developer proposition Apple has presented us. They get to market it as Apple Intelligence, you get the blame if it goes awry.
Apple plans to scale up its News app by adding new countries to the platform beyond the US, Canada, the UK, and Australia, according to the Financial Times.
The plans reportedly include building its locally focused news coverage in the UK, as well as bringing its puzzles section to the country which is currently limited to the US and Canada.
With Apple News, Apple does have access to the full article text. Maybe it will use this to dogfood a way of making this available for notification summaries.
Previously:
- Misguided Apple Intelligence Ads
- Apple Intelligence in macOS 15.2 and iOS 18.2
- Apple Intelligence in macOS 15.1 and iOS 18.1
- Beta for Apple Intelligence in Apple Mail
6 Comments RSS · Twitter · Mastodon
Once in a while I get it right: https://mjtsai.com/blog/2024/06/10/apple-intelligence-announced/#comment-4104609
I don't get what problem AI concocted notification summaries are solving.
Hell of a risk to their reputation Apple have knowingly taken for little or no value.
The problem Apple tried to solve was that more and more people were saying "Apple are late with AI. They are missing the boat. They are old and slow"
So they clearly had a bunch of workshops where mostly marketing people were tasked with integrating "ai" into iOS in as many ways as possible. Few, if any, developers had a say.
Similar to when people were laughing at their skeuomorphic design language and they rushed their new flat look out the door.
I wish Apple had held fast and said "When there is something that will actually benefit our users we will launch it." But 'the market' needed action!
And then there's that stupid fucking name they had to put on it.
There's so much that's right about your comment. I have rarely read remarks on how really ffff dumb 'apple intelligence' is as a name. The Apple presentations are already really weird, weird people, twisting language, weird tone of delivery - seeing 'apple intelligence' up on the screen was hilarious.
https://mjtsai.com/blog/2025/01/07/apple-intelligence-news-notification-summaries/#comment-4219919
Yeah I get why they are rushing out AI features but why summarise headlines?
Headlines are already summarised. There's nothing to be gained and a high chance of losing trust.
One of many categories that should be left untouched by the random word-salad generators.
I think the sad truth is that there aren't that many every day use cases where "ai" would be beneficial. So Apple chose to cram in a bunch of useless crap. Much like when Google added a generate a background image in android two years ago.
It's desperation.
If they ever make "ai" dependable enough to actually do the boring shit like trawling through all emails and put together a nice summary of hotel and flight bookings for a trip, or add reminder for the ever changing activities I need to take the kids to then we're talking.
I'd *love* it if the OS could scan my incoming emails from my sons diving class and update the family calendar with changed times, cancelled lessons, upcoming competitions and etc.
But I'm not holding my breath. Instead we get Apple generating fake news and calling t intelligent.
