Video Conferencing Apps Send Audio When Muted
Jason Daley (PDF, via Tristan Greene):
They used runtime binary analysis tools to trace raw audio in popular videoconferencing applications as the audio traveled from the app to the computer audio driver and then to the network while the app was muted.
They found that all of the apps they tested occasionally gather raw audio data while mute is activated, with one popular app gathering information and delivering data to its server at the same rate regardless of whether the microphone is muted or not.
The researchers then decided to see if they could use data collected on mute from that app to infer the types of activities taking place in the background. Using machine learning algorithms, they trained an activity classifier using audio from YouTube videos representing six common background activities, including cooking and eating, playing music, typing and cleaning. Applying the classifier to the type of telemetry packets the app was sending, the team could identify the background activity with an average of 82% accuracy.
Previously:
Update (2022-04-27): See also: Hacker News.
Update (2022-04-29): See also: Bruce Schneier.
Update (2022-05-09): Rogue Amoeba:
With SoundSource’s Input muting, you can be certain your mic is muted.
Previously:
10 Comments RSS · Twitter
It is obvious that Zoom (and presumably other apps) are still listening when “muted” since they typically detect things like you starting to speak and display a message on screen to remind you that you are muted.
Obviously, sending this audio down the wire is a breach of privacy/trust - which is entirely unshocking.
I installed Webex for a seminar the other day, and then a day later it randomly asked me for permissions to use the microphone. I uninstalled it at that point.
When this was on Hacker News someone pointed out that pausing the audio stream would potentially cause bandwidth issues when unmuting.
@Anonymous Seems to me that the orange dot is more for apps that you don’t expect to use the mic most of the time. For a video conferencing app, it doesn’t really tell you anything actionable. It could be orange because the mute is implemented in software (so it can warn you if you forget to unmute yourself). It also doesn’t tell you whether the data is being uploaded.
For true muting use an app like Mic Drop. With that, Zoom receives no audio until I hit my hotkey to unmute in Mic Drop.
Audio Hijack + Rogue Amoeba Loopback is another solution. Make an Audio Hijack session that points to the loopback device. Select the loopback virtual mic in your video conferencing app. It won’t receive audio until the corresponding session is started. Of course you can also use control blocks in Audio Hijack to turn on/off audio, and even script it.
As much as I understand your overall concern I fail to understand why you think MUTE means "does not listen".
Besides all the dark reasons you suggest EVERY audio conferencing system, no matter if in hardware or software listen to the mics even when muted.
Reason is, that the AEC = audio echo cancelling process constantly needs to hear what is coming from the remote side.
So MUTE simply means, not sending anything to the receiver, but does not mean unplugging the mic.
AEC should not be needed when muted.
It's to prevent the microphone picking up the output of the speaker.
Basically the speaker outputs a sound. The microphone records it. You run auto-correlation to figure out the delay between these 2 signals. Then you remove the sound from what the microphone records before sending it down the wire. If the microphone is off, there is nothing to send down the wire, and no autocorrelation to do.
The delay shouldn't vary that much, unless you're moving around in a place full of (loud) echoes, so keeping track of it all the time is unnecessary.
Sure, if one keeps track of the delay(s) constantly, one can make for a smoother experience, but the problem is that requires trusting that the corporation that made the closed source product one is using will not abuse that trust. A quick google search should disabuse you of any remaining naivety you might have on that score.
Issues "pausing" the audio stream are a red herring. You don't have to pause is. You just need to attenuate it to 0 and transmit silence. Not rocket science.
>As much as I understand your overall concern I fail to understand why you think MUTE means "does not listen".
That's absolutely what it should mean.
I get that there are edge cases (such as Teams not wanting to properly mute so it can instead, ironically, show a "hey, you're on mute but saying something" popup), but that doesn't change the overall user expectation.
> Reason is, that the AEC = audio echo cancelling process constantly needs to hear what is coming from the remote side.
There is nothing to cancel when the mic is on mute.
>Issues "pausing" the audio stream are a red herring. You don't have to pause is.
>You just need to attenuate it to 0 and transmit silence. Not rocket science.
In many codecs this would cause a special tiny "silence" packet to be emitted instead of a bunch of wasteful ones containing zeros.
