Audio Intelligence
Audio Intelligence is a brand-new class of Apple Intelligence features that use the microphones of Apple Watch, the power of Apple silicon, and on-device and cloud-based AI models to help you detect sounds, identify music, and stay present and engaged — all while protecting your privacy at every step.
- Siri Recap: Uses ambient listening to generate high-level notes from conversations, such as takeaways from a team meeting or a parent-teacher conference, producing an Apple Intelligence-generated title, summary, and key points reviewable in the Siri app.
- Live Rewind: Lets users “go back in time for 15 seconds,” transcribing and displaying what was just said in a conversation with a double press of the Digital Crown.
- Shazam: Automatically identifies music playing nearby and surfaces the song title and artist on the Music Recognition widget in the Smart Stack.
- Sound Recognition: Brings sound alerts to Apple Watch for users who are deaf or hard of hearing, listening for sirens, alarms, doorbells, a baby crying, and more, even when their iPhone isn’t nearby.
Apple is emphasizing that all of them are built to prioritize privacy and security using the extensive infrastructure the company has developed to protect its other Apple Intelligence and Siri services. As more and more of these types of features debut, though, the audio intelligence announcements are a reminder that AI-powered tools are becoming increasingly universal in all areas of computing and life.
“These features do not create or store audio recordings, and raw audio used for processing is completely inaccessible to the operating systems, apps, the user, or Apple,” the company wrote in a report shared with WIRED.
[…]
Apple emphasizes that its on-device capabilities for Apple watches have expanded thanks to the company’s new S11 chips. The chips include a special memory-protected space that Apple calls the Secure Exclave. Designed specifically for sensor data, the exclave is an isolated buffer that is inaccessible to the rest of the operating system where Apple Watch can hold and process audio data in a protected space.
[…]
To try to mitigate blatant privacy violations, Apple says that the watch produces an audible chime when a watch owner initiates a Live Rewind transcript to alert other people in the area—even if it is on silent mode or paired with headphones. And the features are built to omit or filter out potentially identifying information about speakers. But the sheer scope and breadth of the tools is extremely noteworthy.
If you want to really take a deep dive on privacy and the new features, Apple has published an 11-page privacy paper that goes even further.
Previously:
Update (2026-09-11): Dr. Drang:
Ron Huang introduced a couple of features of the Series 12 and Ultra 4 that struck me as both disturbing and possibly illegal. They certainly seem at odds with Apple’s commitment to privacy.
[…]
I’m sure Apple will argue that there’s no privacy issue because the recordings aren’t being saved. But if there’s a transcription, isn’t that a type of saving?
And then there’s the question of legality. A great variety of law, both in the United States and across the world, concerns the recording of electronic and in-person conversations. I’m sure Apple has considered this, but I have to think some jurisdictions will frown on Live Rewind. If Apple has to defend it, that’s fine. What’s one more squadron of lawyers to Apple? But if an Apple Watch owner has to defend a transcription made innocently—made with Apple’s encouragement, in fact—that seems like an unfair burden. Maybe Apple should indemnify all future Watch buyers from privacy-related legal action.
[…]
The hoops Apple jumped through to avoid certain words is pretty funny. […] You think your watch records the audio around you? No no no. The audio flows into the Secure Enclave, and flowing is definitely not recording because it’s a different word.
Live Rewind and other Audio Intelligence features do not generate audio recordings. Audio flows into the Secure Exclave, is processed, and then deleted.
According to Apple, Live Rewind text snippets “disappear permanently” from the Apple Watch approximately 30 seconds after the Apple Watch display dims, unless the info is saved to the Siri app.
But good luck explaining that to anyone around you after you read back an exact transcript of what they just said. Apple says they should be sufficiently notified Live Rewind is active because “an audible chime plays from the speaker on your Apple Watch, even if your Apple Watch is on silent or you have headphones connected”, plus the watch plays an animation, but it is not clear to me when this happens. It is not shown in Apple’s video. The impression I get is that Live Rewind is always running in the background and these notifications are only played after you double-click the Digital Crown to look at the text.
This feels like a glimpse of how Apple will market its inevitable wear-anywhere glasses product. I do not think it does enough to assuage privacy concerns. I think normal people will continue to react negatively when you tell them you have been passively recording them because, outside Silicon Valley, that is considered gross and invasive.
I have to admit that the Siri Recap stuff all being “documented” using AI slop is really giving me confidence in the level of care that went into designing this feature
I’m frankly surprised Apple even went through with this feature at this time, considering the bans and backlash other AI-powered surveillance wearables are getting.
[…]
Apple is keen to evidence its trustworthiness, with on-device AI and Private Cloud Compute and an insistence that it doesn’t record or log audio and transcripts. However, if people think they’re being discreetly recorded without their permission, it’s possible Apple Watch wearers could experience their own slice of the abuse hurled at Meta Glasses wearers.
I… y’know, Betteridge’s law does, as always, apply.
Given that Apple has to warn of this, it’s the surest sign the features shouldn’t be released in this form. Apple repeats the form of the warning I see regularly at my gym… but apparently it isn’t heeding it.
Just called my HSA provider and upfront they say that calls may be recorded and transcribed by AI and then provide no means of opting out. So of course the person who finally answered says I have to give my social to verify my identity.
Update (2026-09-14): See also: Core Intuition and Steven Aquino.
Update (2026-09-17): See also: Accidental Tech Podcast.
I was walking home from dinner here in Philadelphia a few weeks ago with a friend visiting from out of town, talking about this, and he stopped midway between the restaurant and my house and asked me to count how many cameras we were on at that spot. I think we counted seven or eight. Who knows how many we missed? I’m not saying that’s good or bad, it’s just the world today. So too it’s soon going to be with personal devices recording audio. This is inevitable. I’m glad to see Apple ship features like this early, with pretty limited scope, to provide good examples before other companies ship devices that record and save audio continuously throughout the day. I think it’s better for Apple to lead with privacy-minded versions of ambient transcripts and AI-generated conversation summaries than to follow after other products have hit the market with all-day recordings and whole-life transcripts.
[…]
I shouldn’t be able to hide my iPhone on or near someone’s desk, or in a conference room, or in a car, or anywhere else, to get Siri Recap summaries of conversations where I’m not present. Making this an Apple Watch exclusive feature avoids that. Thus, I think Apple Watch can be trusted to “listen” ambiently, all the time while it’s unlocked on the wearer’s wrist, in ways that even an iPhone can’t.
I applaud Apple’s decision to make Siri Recap, Live Rewind, and Shazam features opt-in. In the case of Siri Recap, they’ve taken the extra step of allowing you to schedule when and where the feature is active, allowing you to set parameters such as ‘never at home’ or ‘only between ten and six.’ This is the right way to treat a feature this sensitive.
[…]
So why do they creep me out?
[…]
Audio Intelligence appears to me to be the first case of a always-on feature with some of the qualities of surveillance we’ve seen in other products—an enduring record, an uncertainty on the part of others in the room about what’s happening—that also takes care to minimize the privacy harms that seem to be essentially intrinsic to such products. By doing their homework and showing us the receipts, Apple has taken fears of audio recording or even detailed after-the-fact transcripts off the table. But what they’ve left there is a set of questions about social and even legal norms and expectations. They’ve also created a potential problem for Apple Watch users in the form of a newfound suspicion that every word is being recorded unseen.
15 Comments RSS · Twitter · Mastodon
Even with the Chime to alert others that this is listening, having an AI transcription of a conversation of someone who doesn't consent to it without much more than a Chime that could easily be ignored is worrying.
Apple may claim they are privacy first but it doesn't actually matter if Apple can read the transcript if it was made without knowledge or consent.
Normalization of pervasive surveillance is not a good thing and I am going to start looking askance at anyone who has one of these new watches, I really wish there was an easy way to determine by looking at the watch if it is doing the transcription and summary...
Apple, making the world worse once again and getting praise because of their privacy washing.
@Benjamin I wonder if we’ll see new legislation. The idea that you could have a secret transcription without an audio recording is probably not something that was contemplated with the current consent laws.
For all the words Apple is writing about privacy, buried in the page about this feature is this little tidbit:
> For Siri Recap, Apple Foundation Models running on Private Cloud Compute (PCC) summarize and format the final output.
This means that no matter what they say, literally everything being recorded is then effectively sent to Apple's servers. Not the "raw audio", but the text transcription of what is said around the watch, 24/7. And keep in mind that Apple was recently found to have some big privacy features be completely useless and easily pierced. So it's not unreasonable to assume that this data isn't as private as they want you to believe. And that's on top of that fact that people can choose to store these summaries for longer than the auto-delete intervals, and then who knows what Apple will do with them. Because Apple should be viewed similar to Google now with all the ads they're packing into their services and devices. With how much they track what you're doing all the time.
I might become "that person" who won't talk to people unless they turn off their watch first. Because I have no idea of knowing if any Apple Watch is a new one that is recording me and sending what I say to Apple.
"And then there’s the question of legality. A great variety of law, both in the United States and across the world, concerns the recording of electronic and in-person conversations. I’m sure Apple has considered this, but I have to think some jurisdictions will frown on Live Rewind. If Apple has to defend it, that’s fine. What’s one more squadron of lawyers to Apple? But if an Apple Watch owner has to defend a transcription made innocently—made with Apple’s encouragement, in fact—that seems like an unfair burden. Maybe Apple should indemnify all future Watch buyers from privacy-related legal action."
https://leancrew.com/all-this/2026/09/new-apple-watches-have-ears/
On top of everything that has already been said about privacy, even if I do trust Apple right now, why should I trust Apple in the future? They have long been in the process of burning the good will and trust they had built up, and have reversed their image of being the one tech company that makes high quality products that are free of ads, distractions and dark patterns, all for the predictable reason of increasing year-over-year profits.
When they inevitably run out of other methods of trying to sustain their unsustainable profit growth, what do you think they're going to do with the treasure trove of private user data they're sitting on? And that's assuming they're not already exploiting it.
Audio recording people without their permission is illegal in many US states. That includes California where Apple is based
Why do tech companies keep getting a free pass to break the law when it comes to AI?
The use cases Apple gave for that fifteen second rewind are among the dumbest shit I've seen.
You have a person standing right next to you, who's job it is to help you order food, who can answer questions about every single dish. But instead you're supposed to start fiddling with the tiniest screen you own.
Or book recommendations? Within 15 seconds of a friend telling you about a book they think you'd like you start staring at your watch? WTF is wrong with people?
The interesting thing is that they have been able to spin it in such a way that every tech person I've talked to says "it's giving an alert and it's only 15s". Somehow people completely missed that there is also a feature that summarizes conversation transcripts 24/7. Same confusion in Hacker News discussions, etc.
@Benjamin
> Normalization of pervasive surveillance is not a good thing
We are way past the cut-off point, I reckon.
Somebody or some management team OK'd this as they realised the health updates to the Apple Watch were very weak and they were desperate for a cool new feature.
But launching audio re-wind transcription is incredibly risky re people's ongoing trust in Apple - good luck to them getting a reasonable reaction seeing as so much coverage of Apple is borderline hysterical.
Silicon valley had decided that the best way to monetize society is to record everything that's happening at all times.
Now we're slowly being boiled to death in order to accept this. First 15 seconds of audio. The cameras in all headphones, glasses, doorbells, pendants, children's toys, ovens, everything.
So that we won't have to interact with other humans. So that SV can insert itself as the layer between us an society.
And shit like this is where it start.
Remember how Apple critics used to call Apple a cult, and how most of us as young Apple Fanfolk reacted so vehemently against that characterisation. And then we got older, had our road to Damascus moment where the products we liked were ruined by the nonconsensual advances of a sleazy roofie-toting software update, and we saw Apple for what it was; what its fundamental character, reflecting the characters of the people who work there, is.
And now it wants us to wear a device that records everything we say, and if we ever try to leave, it'll use that against us... it's like having a Scientology audit session on your wrist all day.
Kinda like being a cult, huh...
Oh, well. Seems I have to be that one guy …
I'll admit, when this was announced my first thought wasn't privacy which, yes, I acknowledge is important and should be taken seriously. My first thought was accessibility. Reading what you just missed, or being able to spontaneously summarise thoughts that you or those around you are having in a given moment do sound like useful things to have. And, after all, assuming the baseline for PCC is adequate (look, I'll never be happy with anything less than total local operation, but it's clear Apple do take this shit seriously), having undifferentiated, unattributed transcripts being handled doesn't seem like the most invasive thing in the world. Even if, yes, you have to wonder how much harder it would be to just say "Pardon?" a bit more regularly, or take notes. They've even got options for triggering summaries based on location, which again suggests that responsible people can avoid snarfing up completely irrelevant conversations …
So I dunno. All I'm saying is that, notwithstanding a terrifying real-world recap of Black Mirror's "An Entire History of You", this has real applications. At the very least, having a decent microphone on the watch makes for better note-taking on the fly.
[…] The new Apple Watches have a feature called ‘Live Rewind’ that allows you to transcribe the last 15 seconds of whatever has been said around you. I’m so torn by privacy-invasive features like this; on the one hand I want to rail against them and make them socially unacceptable, but on the other hand it feels inevitable that most people will want these features or just leave them turned on by default, so any campaign against them is futile. Michael Tsai has more on this. […]
