Thursday, September 26, 2019

What Time Machine Doesn’t Back Up

Apple:

You can use Time Machine, the built-in backup feature of your Mac, to automatically back up all of your files, including apps, music, photos, email, documents, and system files.

Maxwell Swadling:

Time Machine backups skip a bunch of files. Some you might expect, some might surprise you!

[…]

The Voice Memos iPad App On The Mac seems to exclude the one bit of data it is responsible for saving... recordings...

[…]

I covered in another blog post that the key Photos Photo Library database file is missing[…] Missing this and all its associated files needed to restore Photo Libraries without data loss.

I don’t use Photos enough to know what its database stores that isn’t in iCloud Photo Library. But print projects definitely don’t sync using iCloud, and they can represent a huge time investment. It looks like some project-related files do get backed up—maybe Photos is able to rebuild them into a new database?

But I would feel more comfortable with everything backed up and not relying on a potentially lengthy or buggy rebuild. Some other backup software will exclude files by default, but you can adjust the settings to manually include them. That’s not possible with Time Machine.

Maxwell Swadling:

I was genuinely surprised iCloud Drive wasn’t backed up because I think of time machine as having the “go back in time a year and see what the file was” feature, which doesn’t work on iCloud Drive files!

This sounded really bad to me, because iCloud Drive files don’t necessarily get stored in the cloud, either. Your Mac may not currently have a network connection, and sometimes uploads get delayed or wedged. Once in the cloud, the files are subject to beta bugs, and you have only a very limited ability to restore the latest version of individual files that it thinks you deleted.

But Howard Oakley says iCloud Drive does get backed up, and I do, in fact, see the files in my Time Machine backups.

Maxwell notes that Time Machine doesn’t back up folders which are found in iCloud Drive, but in my experience it’s not as simple as that. The rule seems to be that Time Machine does back up the contents of your current iCloud Drive, but only those items which are currently stored locally. Any items which have been evicted to iCloud, and are only represented by local stub files, aren’t backed up.

[…]

For Time Machine to be able to back up a file evicted to iCloud Drive, that file would need to be downloaded to local storage first. Imagine having a 512 GB internal SSD, and over a terabyte of documents in your iCloud Drive, almost all of which were evicted from local storage.

This makes sense and is probably the only reasonable thing for Time Machine to do. But, to me, it’s yet another reason to avoid the Optimize Mac Storage option. If your files in iCloud Drive are important enough to sync between devices, they’re probably important enough that you want to have your own backups of them.

Previously:

Update (2019-09-27): Jesse Squires:

Now that we sort of understand the layout of ~/Library/Mobile Documents/, where the hell are our iCloud Drive documents stored? Those live in com~apple~CloudDocs/. If you cd there, you should see all of the “custom”, non-app-specific files that you’ve stored in iCloud Drive. These should match what is viewable in Finder.

This is the directory that we want to backup. We can use rsync to do that. (Side note: the way that rsync handles paths is a bit odd. It doesn’t like relative paths, or ~, or escaping spaces in directory names. Thus, this script uses absolute paths with spaces.) You just need to fill-in the USER and DEST variables.

Update (2021-09-08): See also: Howard Oakley.

8 Comments RSS · Twitter

I guess nobody bothered to actually test these claims.

Photos does keep a separate set of journal files that it backs up instead of the main SQLite database. I presume this is largely so that it doesn’t have to back up a potentially 100MB+ file every time one small change is made to the library. The first time you open a library after restoring it from Time Machine, Photos does rebuild the database from the journal files.

@Brian Yes, I see why they do that, and I love the idea of being able to rebuild a database, but I’m not crazy about this being the backup strategy. There is no guarantee that at any given time the current set of journal files can accurately reconstruct the current database. Some could be out-of-sync, damaged, missing, etc. Photos could run into a bug or error trying to read them in, and then you have nothing.

The Apple Mail and Lightroom approach seems more robust because the metadata is saved on the actual message and photo files that you can work with, rather than opaque journals. So you can rebuild from an incomplete set of data. (And the Lightroom database gets backed up both to Time Machine and ZIP archives.)

I’ve also seen claims that not everything in the Photos database can be rebuilt. Do you know?

I have just started using TimeMachine, backing up to a Buffalo external hard drive. On my MacBook Pro i have already fitted a Transcend memory card because of the poor/small built in memory. Will the content of the card be backed up by TimeMachine?

Sören Nils Kuklau

@ John: by default, no ("internal" disks only). If you always have the card inserted, try going to System Preferences, Time Machine, clicking Options… and see if the card is listed in "Exclude these items". If so, remove it from there, and TM should back it up in the future.

If you often find yourself removing the card, Time Machine isn't a good fit for it.

Stephen Harling

I've just talked to Apple UK and found out that Time Machine doesnt backup Emails.

It does backup POP Emails but not IMAP which Yahoo and Apple Mails both use.

@Stephen Right, you can’t count on Time Machine to back up IMAP e-mails because Mail doesn’t necessarily store all the messages on the Mac. Even the messages that are stored may not have their attachments downloaded. EagleFiler can help you archive a copy of your IMAP messages that’s fully offline and will get backed up by Time Machine.

Leave a Comment