Apple Intelligence News Notification Summaries
Apple Intelligence, launched in the UK earlier this week, uses artificial intelligence (AI) to summarise and group together notifications.
This week, the AI-powered summary falsely made it appear BBC News had published an article claiming Luigi Mangione, the man arrested following the murder of healthcare insurance CEO Brian Thompson in New York, had shot himself. He has not.
A news summary from Apple falsely claimed darts player Luke Littler had won the PDC World Championship - before he even played in the final.
The incorrect summary was written by artificial intelligence (AI) and is based on a BBC story about Littler winning the tournament semi-final on Thursday night.
Within hours on Friday, another AI notification summary falsely told some BBC Sport app users that Tennis great Rafael Nadal had come out as gay.
The ads for Apple Intelligence have mostly been noted for what they show, but there is also something missing: in the fine print and in its operating systems, Apple still calls it a “beta” release, but not in its ads. Given the exuberance with which Apple is marketing these features, that label seems less like a way to inform users the software is unpolished, and more like an excuse for why it does not work as well as one might expect of a headlining feature from the world’s most valuable company.
[…]
Apple has also, rarely, applied the “beta” label to features in regular releases which are distributed to all users, not just those who signed up. This type of “beta” seems less honest. Instead of communicating this feature is a work in progress, it seems to say we are releasing this before it is done. Maybe that is a subtle distinction, but it is there. One type of beta is testing; the other type asks users to disregard their expectations of polish, quality, and functionality so that a feature can be pushed out earlier than it should.
[…]
This all seems like a convoluted way to evade full responsibility of the Apple Intelligence experience which, so far, has been middling for me. Genmoji is kind of fun, but Notification Summaries are routinely wrong. Priority messages in Mail is helpful when it correctly surfaces an important email, and annoying when it highlights spam. My favourite feature — in theory — is the Reduce Interruptions Focus mode, which is supposed to only show notifications when they are urgent or important. It is the kind of thing I have been begging for to deal with the overburdened notifications system. But, while it works pretty well sometimes, it is not dependable enough to rely on.
I don’t think that the vast majority of people know what beta means. Apple has been promoting the shit out of these features, and putting beta in a footnote.
Xe Iaso (via Hacker News):
This phrases a literal scam message in ways that make me think immediate action is required. You can see how this doesn’t scale, right?
[…]
Even more, if you have Apple Intelligence enabled for some of the other features but disable notification summaries because you find them worthless, you can get your notifications delayed up to five seconds. It’s kind of depressing that telling your computer to do less work makes the result take longer than doing more work.
Additionally, none of the summarization features work on my iPhone and I can’t be bothered to figure out why and fix it. I personally don’t find them useful. I just leave them enabled on my MacBook so that notification delivery is not impacted.
[The] whole vibe of Apple Intelligence is off-putting and feels like a not-ready-for-primetime suite of features that make the user experience worse.
Apple is working on an update for Apple Intelligence that will cut down on confusion caused by inaccurate summaries of news headlines, Apple told BBC News. In a statement, Apple said software coming soon will clarify when notifications have been summarized by Apple Intelligence.
[…]
There have been several prior events where Apple Intelligence provided incorrect details from incoming news app notifications. In November, Apple Intelligence suggested Israeli Prime Minister Benjamin Netanyahu had been arrested, incorrectly interpreting a story from The New York Times.
[…]
Apple Intelligence notification summaries are an opt-in feature and they can be disabled.
My understanding is that they are opt-out in that once you opt into Apple Intelligence in general, you have to opt out of the notification summaries if you don’t want them. And, crucially, this is at the user level. There is no way for an app developer such as the BBC to prevent its app’s notifications from being summarized.
Apple is promoting the hell out of Apple Intelligence to consumers, and its advertisements hide, rather than emphasize, its “beta” quality.
The promotion of a feature is an implicit encouragement to, you know, actually use it.
[…]
Apple Intelligence notification summaries are marked with an icon/glyph, sort of like the “↪︎” Unicode glyph with a few horizontal lines to suggest text encapsulated by the arrow — a clever icon to convey an abstract concept, to be sure.
The meaning of that icon/glyph is not at all obvious unless you know to look for it, and most users — even those who opted in to Apple Intelligence understanding that it was “beta” and might produce erroneous results — don’t know to look for that particular glyph.
[…]
I can also see why Apple doesn’t want to offer such an option to developers. To whom do notifications belong — the developer of the app that generates them, or the user who is receiving them?
The statement uses the beta tag it has placed on Apple Intelligence features as a shield, while promising to add a warning label to AI-generated summaries in the future. It’s hard to accept “it’s in beta” as an excuse when the features have shipped in non-beta software releases that are heavily marketed to the public as selling points of Apple’s latest hardware. Adding a warning label also does not change the fact that Apple has released a feature that at its core consumes information and replaces it with misinformation at a troubling rate.
Apple is shipping these AI-based features rapidly, and marketing them heavily, because it fears that its competitors so far out in front that it’s a potentially existential issue. But several of these features simply aren’t up to Apple’s quality standards, and I worry that we’ve all become so inured to AI hallucinations and screw-ups that we’re willing to accept them.
[…]
So what can Apple do now? A non-apology and the promise of a warning label isn’t enough. The company should either give all apps the option of opting out of AI summaries, or offer an opt-out to the developers of specific classes of apps (like news apps). Next, it should probably build separate pathways for notifications of related content (a bunch of emails or chat messages in a thread) versus unrelated content (BBC headlines, podcast episode descriptions) and change how the unrelated content is summarized.
I side with Apple in not giving developers the option to opt out of notification summaries, and (b) that I’m a bit more of the mind that Apple can address this by somehow making it more clear which notifications are AI-generated summaries. Like, perhaps instead of their “↪︎” glyph, they could use the 🤪 emoji.
If Apple Intelligence summarizes your notifications then Apple should badge it with their Apple logo. Not some weird cog or brain or some other such icon. Put your name on it! Apple is the one presenting this information to you and they should be held accountable for the veracity of it. Put your highly regarded Apple logo on your AI work or get outta here. It’s either an Apple product or it’s not.
The problem with Apple’s approach is that it’s summarizing a headline, which is itself a summary of an article written by a human being. As someone who has written and rewritten thousands of headlines, I can reveal that human headline writers are flawed, some headlines are just not very good, and that external forces can lead to very bad headlines becoming the standard.
Specifically, clickbait headlines are very bad, and an entire generation of headline writers has been trained to generate teaser headlines that purposefully withhold information in order to get that click.
[…]
Summarizing summaries isn’t working out for Apple, but more broadly I think there’s something to the idea of presenting AI-written headlines and summaries in order to provide utility to the user. As having an LLM running all the time on our devices becomes commonplace, I would love to see RSS readers (for example) that are capable of rewriting bad headlines and creating solid summaries. The key—as Artifact learned—is to build guardrails and always make it clear that the content is being generated by an LLM, not a human.
Starting to think Apple might regret sticking its name in front of ‘Intelligence’ for all its AI stuff. Notifications are a disaster. Image Email categories are a disaster. And so on. Then again, the ad campaign is somehow even worse than all of that.
The sad thing is, there are good elements to Apple AI/ML. Prompt-based memories in the Photos app. Auto-tagging. Accessibility features like Personal Voice. But so much attention has been grabbed by flashy stuff that did not – and in some cases could not – work.
The Apple Intelligence vs BBC story is a microcosm of the developer story for the feature. We’re soon expected to vend up all the actions and intents in our apps to Siri, with no knowledge of the context (or accuracy) in which it will be presented to the user. Apple gets to launder the features and content of your apps and wrap it up in their UI as ‘Siri’ — that’s the developer proposition Apple has presented us. They get to market it as Apple Intelligence, you get the blame if it goes awry.
Apple plans to scale up its News app by adding new countries to the platform beyond the US, Canada, the UK, and Australia, according to the Financial Times.
The plans reportedly include building its locally focused news coverage in the UK, as well as bringing its puzzles section to the country which is currently limited to the US and Canada.
With Apple News, Apple does have access to the full article text. Maybe it will use this to dogfood a way of making this available for notification summaries.
Previously:
