Chrome Updater Bug Prevents Macs From Booting
Tim Hardwick (Avid, Hacker News):
Variety reports this morning of a possible computer virus attack or critical software failure affecting Mac Pro workstations across Los Angeles.
According to social media chatter, Hollywood Film and TV editors discovered late on Monday that “trashcan” Mac Pros running older versions of macOS and AVID’s Media Composer software were refusing to reboot after shutting down.
After further investigation it was found that AVID was not the problem!
[…]
After investigation from some of the top minds in the MacAmins Slack Chat #varsectomy channel it was found that the Google Keystone Updater was at the heart of the issue.
We recently discovered that a Chrome update may have shipped with a bug that damages the file system on macOS machines with System Integrity Protection (SIP) disabled, including machines that do not support SIP. We’ve paused the release while we finalize a new update that addresses the problem.
[…]
To recover a machine that has been affected by this bug, please boot into recovery mode, and then from the Utilities menu open the Terminal application.
In the Terminal application, you can run the following commands[…]
The now-pulled Keystone update attempts to remove the /var symlink, which is usually protected by Apple’s System Integrity Protection (SIP) security feature.
On Macs where SIP was disabled, this protection did not apply and the Keystone update was able to remove the /var symlink. This symlink is not a directory itself, but points to another directory (/private/var) which contains software necessary for the operating system to boot and function correctly, so removing the /var symlink rendered the affected Macs unbootable.
Update (2019-09-26): Jeff Johnson:
Something fishy with Google’s latest comment. Seems to be shifting the blame. Why act as if the updater doesn’t have root?
Why in the world would a web browser’s software updater be doing anything at all at the root level of the boot volume? The arrogance and presumptuousness here boggles the mind. This is like hiring someone to wash your windows and finding out they damaged the foundation of your house.
The other question is why in the world so many users would disable System Integrity Protection. The answer seems to be that it’s the only way macOS will let the AVID customers use third-party video cards.
See also: Hacker News.
Update (2019-09-27): Jeff Johnson:
People: Why does a web browser installer need to modify the system?!?
Me:
$ lsbom /System/Library/Receipts/com.apple.pkg.Safari13.0.1MojaveAuto.bom | grep /System/
The Google Keystone bug isn’t a justification for System Integrity Protection. In fact, if SIP didn’t exist, Google would most likely have noticed the bug before shipping it. So in a sense, SIP is partially to blame for the disaster.
This is true, but it doesn’t mean SIP was a bad idea. Rather, SIP is treating the symptoms rather than helping to identify the causes. It certainly could do more of the latter, e.g. if it maintained an audit log. I don’t mean the gigabytes of console spew that we currently get for SIP and sandbox violations. Instead, there should be a friendly window that concisely shows what each app was thwarted from doing. The Chrome developer—or even Chrome users—would be able to see at a glance that it tried to delete the /var folder 39 times and would then be able to ask why.
Every app outside the Mac App Store has to roll its own software updater. This is how we get software update problems. Apple has left this gaping hole in the system forever. Why is there no system process and API for 3rd party app updates?
It’s a totally obvious idea that could have been done 20 years ago. And it would be more helpful today in that updating sandboxed apps is harder. But it’s also kind of a strategy tax. Making life better for directly sold apps (and their users) would cost services revenue and reduce the value proposition of the Mac App Store.
Update (2019-10-13): To be clear, the Chrome updater only asked for root access if you enabled the option to Automatically update Chrome for all users.
4 Comments RSS · Twitter
Tangentially related: I’m not sure “#varsectomy” promotes the kind of inclusive environment we’d like to see in software.
The Chrome bug is insane. That is all.
Google pushed this Keystone Updater across all skus, I noticed it when BlockBlock threw up a dialog for no reason whatsoever, on my 2012 Mini.this would’ve crippled my Mini too.
SIP is annoying but NEVER turn off SIP. Those Avid jockeys should run Windows until Avid properly signs their drivers.
Google is now malware, they’ve crossed the malware line. No updater should ever need root. Google should FIRE and BLACKLIST that installer “engineer.”
EVERY MAC USER should run BlockBlock. All the positives with none of the negatives of Catalina on your macOS now.
Every app outside the Mac App Store has to roll its own software updater. This is how we get software update problems. Apple has left this gaping hole in the system forever. Why is there no system process and API for 3rd party app updates?
No, no, no.
Yes, it would have been great if Apple had shipped a standard way to update apps (and also a better mechanism to remove them that also removes satellite files), but Sparkle does exist, has existed for a long time, and gets the job done just fine.
Google could’ve used that. They do not want to, because of NIH culture, and also because they don’t want the user’s consent in changing their software whenever they please (something euphemistically known as “evergreen” now).
Wait, the Chrome updater deleted /var?
Look, as someone who just deleted both my anacrontab and anacrontab.bak file because I was a lazy typist (just cp first file to second file location and then changed my command to rm, instead of retyping, stupidly autocompleted and clicked enter without thinking…yeah, that kind of stupid), I get how these things can happen.
I even had a Samba problem when my fstab was edited to map a Samba drive at boot and the file browser kept freaking out when I navigated to the mount point. I even copy and pasted the correct entry from a different fstab, well correct, except for the specific mount point on the server side, so I edited the share manually and saved. After going on my merry way, none the wiser, turns out when I deleted a bit of the IP address while editing the share, instead of undoing that deletion, I got cute and manually retyped the address before saving the fstab. Wrongly retyped as it were when I discovered the last two digits of the IP address were transposed! Oops! E.g. /10.0.0.1/fooshare became 10.0.1.0/fooshare.
Even still, Google should be able to do better than Mr One Man Band, home IT guy like myself. Geeze!
