Tuesday, July 20, 2010

Arq

Arq is an application for backing your Mac up to Amazon S3 (via Matt Henderson). I’m happy with CrashPlan, but Arq promises advantages such as better support for metadata and direct access to your backup data in a documented, Git-like format. S3 has an API and many clients, and Amazon is likely a very stable host, but it’s also considerably more expensive if you have hundreds of gigabytes of files.

5 Comments RSS · Twitter

"I’m happy with CrashPlan, but Arq promises advantages such as better support for metadata and..."

The whole reason I originally bought DropDMG was to be able to protect stuff like metadata in my files and locally encrypt them in an image for online backup.

Now I've evolved to an offsite sneakernet setup. Bare drives are cheap and just lying around. Clone images are sitting there on my server. And the once a month task of remembering to bring the drive home, stick it in a drive dock, and have all of my clones automatically copied over by CCC is a small enough effort to make me want to keep the $400/yr I'd otherwise be paying Amazon.

But if prices were to fall, and/or I was on the road more often, and/or I wanted to increase the frequency of my offsite backups, I'd be tempted to return to online backup with an Arq/S3 solution. I genuinely love being an Amazon customer. Amazon has always been eager to appease me as a customer when issues arise, and that leads me to have a high level of trust in them going forward.

And while I'm not familiar with the Arq developer's work, I do like his reasoning in his Arq overview.

I use CrashPlan as a third line of defense after off-site clone images (created with DropDMG, natch) and Time Machine. It provides high-frequency off-site backups, backs up when I’m traveling, and stores versions that go farther back than Time Machine. I can do without the metadata in this particular case.

Arq seems to be designed the Right way, and I can’t think of a company I’d trust more than Amazon, so I find this an interesting solution even though I have no plans to switch.

Some things I find interesting about Arq are:

1.It uses Content Addressable Storage (CAS), so that content is uniquely identified by a hash, preventing stored duplicates (and redundant costs). CAS is also used by Dropbox (and Backblaze); if you add a common software installer DMG to Dropbox, it'll appear to upload nearly instantly, because somebody somewhere has already uploaded it. Of course, in a local context, one probably won't have that much duplicated data, but the fact that the Arq author is using CAS gives me confidence in the quality of his product.

2. Arq offers the option to use Amazon's "Reduced Redundancy" storage class, which cost about 30% less than normal S3 storage. I've told Arq that my budget is $25/month, which gives me 250GB of storage at Amazon's Reduced Redundancy storage (at $0.10/GB). It'll handle archives like Time Machine until that storage is used, and then trim older archives accordingly. I also expect (if the past is any indication of the future) that Amazon's prices will continue to drop over time.

Anders Thoresson

How are you using DropDMG for backups?

@Anders I make nightly clones using SuperDuper. The most recent couple clones I keep on-site. Every week or so I use DropDMG to create an encrypted and compressed disk image of the clone. The compression lets me fit multiple clones on a single drive, and the encryption lets me store it off-site safely. Because the disk images are checksummed, I can verify when restoring that the backup is good.

I also use DropDMG to make encrypted, compressed, and segmented disk images of certain key folders and burn them to DVD.

Also of note: CrashPlan now supports extended attributes.

Leave a Comment