Friday, September 15, 2017

SuperDuper and APFS

Dave Nanian:

The bad news is I’m not confident enough to say we’re going to release our APFS support day-and-date.

I know this kind of hedging is disappointing. But it’s important to note that Apple still hasn’t released any documentation on the “proper” way to create a bootable APFS volume. An example of what they have in mind was released for the very first time when the High Sierra developer release came out a few months ago, but that’s it. We basically have to make an educated guess about what they want.

We’ve designed and implemented that, and it’s significantly different than HFS+’s boot setup, with various special partitions dedicated to specific purposes (even a separate VM volume!), and entire new volume management system, etc.

[…]

For example, what happens if you do an “Erase, then copy” from an HFS+ volume to an APFS volume? In our current version, we match the format of the source when we erase. But, HFS+ can’t be in an APFS container. So, we’d have to convert the container to a regular GUID partition. And since there might be other APFS volumes in that container, you’d end up destroying them.

[…]

In particular, Apple has further tightened its System Integrity Protection process, and is completely denying access to some files on the startup volume, even when copying to a non-startup volume.

[…]

APFS doesn’t seem to be faster than HFS+ (which is not to say it won’t ever be, or that it won’t be more stable...a low bar, I know).

Mike Bombich:

Apple offers a couple helpful APFS-related knowledgebase articles here:

Apple Kbase HT208018: Prepare for APFS in macOS High Sierra
Apple Kbase HT208020: Upgrade macOS on a Mac at your institution

In regard to how CCC will work with your APFS-formatted volumes, this CCC knowledgebase article aims to answer all of the questions you might have on the subject:

Everything you need to know about Carbon Copy Cloner and APFS

Previously: macOS 10.13 High Sierra Shipping Soon, Pondering the Conversion From HFS+ to APFS.

Update (2017-09-20): Alastair Houghton:

Now, in the case of macOS 10.13, there is a bigger problem. Apple is changing filesystem. In order for a low-level disk utility like iDefrag or iPartition to function, we need to know exactly how the filesystem organises data on disk; indeed, inside our products we have pretty comprehensive implementations of HFS+, FAT and NTFS. Apple’s new filesystem APFS, is a completely new design, and you’d have thought that Apple would give us disk utility vendors a fighting chance of getting up to speed before the release of 10.13 by releasing design documentation well in advance, but no, that hasn’t been the case this time around. The only documentation we have about the APFS volume format is this table. Yes, that document includes other information about what APFS can do, but it doesn’t include any detail of the on-disk data format other than a table comparing it to HFS+.

While it’s impossible to be certain, it’s highly likely that adding APFS support to our products, if/when Apple ever releases technical details of its volume format, will involve months of work, and since APFS is going to be the default format for many devices (specifically, anything that uses only Flash storage), as well as being an option for other situations, we simply can’t promise macOS 10.13 support right now.

Update (2017-09-25): Dave Nanian:

We’ve finished up a bunch of internal testing over the past few weeks, and there’s a beta of SuperDuper! for High Sierra and APFS linked at the bottom of this post. But it’s so exciting, in a totally nerdy way, that it would be a mistake to not follow the whole story, with its twists and turns. So let’s dive in.

Update (2017-10-02): Dave Nanian:

Interesting tidbit for the curious: if you turn on encryption, while APFS is converting the drive, snapshots cannot be created.

That means that, during the encryption process, neither Time Machine nor SuperDuper can back up. So, be aware and back up first.

Update (2017-10-17): Dave Nanian:

The introduction of APFS allowed us to revisit that decision. Because its more flexible volume creation is low-impact, the risks inherent in adding and managing the Recovery volume itself are minimal. Recovery now has its own special, documented “Role” within the APFS container, and its contents follow the pattern established for Preboot. Even encryption is done differently: it's properly managed in Preboot, which can be created and updated by a documented system tool, provided by Apple, further ensuring proper operation and compatibility as Apple makes changes and modifies requirements.

After carefully evaluating the new support and determining there were minimal risks, we decided that we could safely copy and manage Recovery for APFS containers, whether copied from APFS or HFS+ sources. And so we do.

Update (2017-11-01): Dave Nanian:

We’ve discovered during our broader Beta rollout that, due to weird bugs in Disk Utility, formatting an HFS+ drive as APFS is unreliable too. Sometimes the drive just “vanishes” and doesn’t re-mount. Sometimes it fails for no reason. Sometimes it makes the one volume unreadable until it’s erased again.

[…]

The problem is, log show -last 1m, including a kernel predicate so it only returns low-level kernel logging, can be slow. Not only that, but due to bugs in the logging subsystem, it can incorrectly return much more than one minute of logging. We've seen it return almost a gigabyte of log data!

Update (2017-11-09): Dave Nanian:

SuperDuper! 3.0 has, literally, many hundreds of changes under the hood to support APFS, High Sierra and all version of macOS from 10.9 to the the present.

SuperDuper! 3.0 is the first bootable backup application to support snapshot copying on APFS, which provides an incredible extra level of safety, security and accuracy when backing up. It’s super cool, entirely supported (after all, it’s what Time Machine uses...and it was first overall), and totally transparent to the user.

4 Comments RSS · Twitter

[…] Previously: SuperDuper and APFS. […]

[…] SuperDuper and APFS, Finder 10.9 Disk Space […]

Leave a Comment