Faster Xcode Updates
Igor Kulman (via Cédric Luthi):
Installing Xcode from the Mac App Store might seem like a convenient way to do so but it is too slow and inflexible. You cannot use the Mac App Store to install multiple version of Xcode at the same time if you need them, like when testing with a Xcode beta for an upcoming iOS release. Download from the Mac App Store is incredibly slow and sometimes not even available for days after release (like 11.2.1).
[…]
Downloading Xcode from the Apple Developer Portal is faster than using the Mac App Store, but it can be made even better. You just need to use the right tools.
[…]
This script downloads the given Xcode by URL from the Apple Developer Portal, but uses up to 16 separate connections to do so. You will see a significant download speed improvement.
Pro tip: if you downloaded Xcode straight from Apple, you can use
xip -x Xcode_12.1_GM_seed.xipto skip the validation step and save approximately a billion tons of CO².
And don’t forget to turn off Dropbox.
Previously:
Update (2022-01-17): Saagar Jha:
Tip: when expanding an Xcode XIP archive, use the command line (xip --expand) rather than Archive Utility. It’s at least 25% faster–sometimes even twice as fast, depending on the circumstances. They both call into the Bom API, so I profiled both to see why there’s a difference.
[…]
Anyways, the decompression process for a XIP is fairly straightforward: read files out from the XIP’s LZMA stream, recompress them with LZFSE if possible, then write them to disk. The compression is handled by AppleFSCompression, and it’s more than happy to parallelize the task.
[…]
Specifically, it’s the creation of those half a million files that really hurts overall performance. The “driver” thread is unable to create files fast enough to keep the worker thread pool busy doing actual work on the CPU, because it’s blocked by file operations in the kernel.
[…]
How expensive is this evaluation? Pretty expensive, it turns out. Filesystem operations might spend up to 30% of their time on CPU just evaluating sandbox policies! And all of these run synchronously on that one thread, so they block everything else from proceeding.
[…]
Secondly, I found that xip consistently spent less time in Sandbox evaluation. I can’t be sure why, but my guess is that it has a simpler profile (it’s basically unsandboxed…) so the policy might be simpler to check. All other things being equal, it’s still 25% faster overall.
Update (2022-02-04): Ken Kocienda:
I don’t think I have ever successfully downloaded Xcode from the Mac App Store. Just failed again. Simple, convenient, but doesn’t work.
7 Comments RSS · Twitter
Regarding the tip to turn off Dropbox:
Is it really still using so much CPU? I didn’t use Dropbox for a long time, but were considering to start using it again.
I've essentially given up on Xcode from the App Store, because something is seriously wrong either with Mac OS, APFS, or my computer:
With 44 GB free, App Store refuses to download Xcode, citing insufficient space: https://twitter.com/zygoat/status/1309562213531754496
Mind you, even after downloading it manually it's often not possible to copy over, let alone unarchive: https://twitter.com/zygoat/status/1312142057054466048
(I've filed bug reports: FB8742389, FB8742506. No acknowledgements or follow-ups received, of course.)
Apparently the xip -x trick only delays the verification, which then happens at first launch, so it doesn't really help.
@Adrian I want the whole Xcode, anyway, because you never know when you’ll need to go back to a previous version.
If there is something I wish they learn from Microsoft, it is to let the user choose what he want to install.
I'm doing Mac only dev on my machine, and I don't want to waste many GB of disk space to store WatchOS, CarOS, iPadOS SDKs and other associated simulators.
@ Jean-Daniel: [I'm a .NET / VS dev]
Honestly, I'd say it's a mixed blessing.
On the one hand, yes, VS has moved to a model with multiple "workloads", where you can significantly reduce the overall footprint (e.g., you can say you're only interested in Web development, not in mobile development).
On the other hand, it also significantly complicates the setup process, leading to pains for both users as well as I imagine MS's own team. Case in point: to install multiple side-by-side copies of Xcode, you just rename the bundle. While VS has moved a _little_ towards being more self-contained, it does still spread plenty of stuff all over the file system and registry. There's a thing to be said for classic Mac-like "you open this file and everything works; no 'installation' step required".
Having said that,
* I would like the compromise where Xcode came just with the bare essentials, and you download one or more "Something.xcodeworkload" bundles that contain the iOS SDK, etc.
* it's become a joke how long Xcode's overengineered cryptographic checksums take to verify for every… single… minor update. I shouldn't need an M1 for *that* reason. More componentization would help this poor UX, too.
TL;DR: meet somewhere in the middle?
