Arq 6
It’s been 4 years since Arq 5 first came out (although we’ve shipped 155 updates to Arq 5 in the meantime).
Our goal with Arq 6 was to make it work with the latest operating system features and the latest cloud storage options, make it more secure, and add the features people have been asking for, all while keeping it very simple to use, with smart default settings.
It’s $50 or $25 to upgrade. Lifetime licenses are no longer for sale, but they’re honored if you had already purchased one.
We worked with Apple to get access to the APFS API, so Arq 6 has native support for creating and using APFS snapshots.
This is great because it means Arq’s backup will be a perfectly consistent view of your files at a point in time, even if you’re modifying files as Arq is backing them up.
Backing up from a snapshot is great, and it’s one of several important enhancements to Arq’s backup engine:
Multiple backups can now run in parallel, so you can keep making regular backups to important small files while uploading your photo archive over a period of weeks.
You can finally give backups and storage locations names. This had been annoying me daily for about 10 years.
It supports Amazon’s Glacier Deep Archive (1/4 the price of Glacier, but with a 12-hour restore delay) and Google Cloud Storage’s Archive (no delay, but can’t delete for a year).
The rate limiting is more flexible and can be set per-backup.
The retention rules are now customizable (and per-backup).
It can warn you if there hasn’t been a backup in a while.
You can restore without having to first download large amounts of metadata, which was previously necessary when restoring from another Mac’s backup or if Arq’s cache had been deleted or lost.
Lots of operations are apparently faster or more efficient, although I haven’t had a chance to experience this yet.
The user interface has been completely redone. The good news is that the new design makes much more logical sense. It is easier to find things and to see what the app is currently doing. You can view the log for a backup right in the activity display, without having to go to a separate app. There are finally progress bars.
The bad news is that the app you interact with is now built with Electron. (The background agent process that does the work remains Objective-C.) The problems with Electron range from the superficial (everything just looks and feels off) to the functional (you can’t navigate outline views with the arrow keys or type-selection).
There are also areas where the new interface is designed in a less useful way. For example, when selecting which files to exclude, there used to be an outline view showing a live view of the file system, with checkboxes to mark items for exclusion. And there was a rule-criterion-like interface for excluding items based on patterns in their names. With Arq 6, this is all replaced by a text box filled with a mix of paths and globs. It’s short and narrow, with no indented wrapping, so I pretty much need to copy the text to BBEdit to see what’s going on. Even then, it’s cumbersome enough that I need to invert my backup selection model: instead of including a few big folders with lots of exclusions, it now makes more sense to selectively include smaller folders.
Other useful features from Arq 5 are absent. It no longer shows the amount of cloud storage used. You have to enter the backup password every time you change a setting. The log and status e-mails no longer tell you what was backed up. You can no longer see a preview of which items will be backed up. You can no longer start and stop backups from the menu bar. You can no longer quit the agent process—I now regularly force-quit it, as it swells to 25 GB of RAM. You can no longer control the software updater, which now updates to the latest version without even telling you. The data format specification and documentation are not available yet.
Overall, it just seems like the app was shipped before it was ready. After upgrading, all four of my backups (1 Amazon, 1 Google, 2 to hard drives) encountered problems importing the old backup data. One import reported 1.7 million errors, which ground the Electron interface to a halt and consumed 5.3 GB of RAM. I ran into a cascade of issues, and none of the original backups is yet working properly. Some backups got stuck and don’t do anything. Others seemingly want to upload all the files again, even though only a few have been modified.
I don’t think I’ve lost any data (though apparently some users have). The import process doesn’t delete the Arq 5 metadata, so it should still be possible to access all the snapshots with Arq 5. And Arq 5 snapshots are also restorable within Arq 6, although I ran into interface glitches with both the snapshot list and the file outline that made it impossible to actually access most of the data because the view wouldn’t scroll.
Still, to be on the safe side, I decided to put my old backups on pause and create all-new ones for Arq 6. The developer is normally very helpful and responsive but seems to be completely swamped making a flurry of fixes. It’s not really possible to get help at the moment, so I don’t want to mess anything up. New backups also make sense, anyway, because it’s the only way to take advantage of the new cloud storage classes, and because of the changes to selection/exclusion.
I’ve fixed a lot of issues in the past 5 days. I think importing is quite stable at this point. I apologize again for the problems. I will keep fixing issues as quickly as possible. And I’m going to rework the UI to look a lot more like Arq 5.
This is a sad situation because Arq is one of my favorite Mac apps of all time, and I don’t like to see it struggle. Plus, as a developer, it’s my nightmare to ship a build that I thought was stable only to find major bugs that no one encountered during testing. And then to have to choose between fixing the bugs ASAP and being responsive to the customers who are hitting them.
I still think that Arq, uniquely, gets the overall backup architecture right. Arq 6 makes major progress, but it also has some serious regressions. Most importantly, it needs more testing and refinement. I do not recommend updating right now. Arq 5 still works great. There’s no rush.
I remain positive about the long-term prospects for the app. There is, unfortunately, a history of major releases shipped with inadequate testing. But the developer has always worked hard to fix everything. Most of the time, Arq has been incredibly stable with everything just working as it should.
Previously:
- What Time Machine Doesn’t Back Up
- Arq Cloud Backup 1.0
- CrashPlan Discontinues Consumer Backups
- Arq 5.9 Adds Backblaze B2 and Wasabi Support
- What Backblaze Doesn’t Back Up
Update (2020-04-15): Stefan Reitshamer:
Unfortunately quite a few people experienced problems with importing Arq 5 settings and data.
I sincerely apologize for that. We’re doing everything we can, as quickly as we can, to fix the issues.
[…]
We’ve gotten a lot of feedback on the UI and have taken it to heart. We’re going to rework the UI to be cleaner, more compact, and laid out more like Arq 5.
I think the new layout is fine. I just prefer having native controls and button ordering.
Update (2020-04-17): Arq Backup:
We’ve removed the feature for importing Arq 5 data into Arq 6 format because of too much incompatibility between the 2 apps’ configuration approaches. Instead we’re working on adding an Arq 5 compatibility mode which will function just as Arq did -- same folder structure etc.
We plan to deliver compatibility mode before end of June 2020.
18 Comments RSS · Twitter
It’s very odd. I was never able to import my one backup from v5 and was never able to successfully do a test backup to an SFTP target. Arq would connect to the server and make a folder but never copy files over. I reported the issue several times the week before the release and then decided to downgrade and leave it alone for a few months.
We would you built a Mac-only (that's what I assume since it's tied to APFS features) app using Electron? 🤔
I used Crashplan for many years, and Backblaze before that. When Crashplan shut down their consumer product I moved to Arq + Wasabi and that has been the best solution so far, by far. It just ran invisible in the background and I had full control over retention, encryption, exceptions and everything else. Also, Arq seemed to be the best way to preserve Mac meta data correctly, of all backup software. Arq had been around for years and users loved and trusted it.
That's why it's so sad to see – and being part of – what has evolved of the last couple of days with the launch of Arq v6. Since I was happy with Arq v5 and had full trust in the developer I immediately upgrades to Arq v6 when it was released. Unfortunately.
There were so many thing that went wrong at the same time.
First, I had to pay an upgrade price even though I previously bought the v5 lifetime license. Nothing of the upgrade process made me aware of my old license. The developer has since repaid my purchase, after I emailed the support. It's was not much money and even though I've paid extra for lifetime I'm actually ok with a small upgrade fee to keep the product alive. But the UX was not great, and many felt deceived by this.
Then it was time for import of old snapshots to preserve backup history and this did not go well. The import process ran for a day or so, and during that time no new backups were being made (kind of makes sense but could have been made clear before the start). Then it stopped as if it was finished (with 180 errors of which I have no indication if they where minor or major), but every time a new hourly backup was initiated it was immediately aborted because ”Import in progress; backup aborted.” So I waited another day but nothing was being backed up. I email support and was asked to install a newer version (which had to be done manually since there where no "check for updates" or any automatic upgrade) and then abort the import.
After upgrading and aborting the import the backups finally started to run. I still didn't really know if the import was ok (since there were errors and I had to abort it).
Then another issue presented itself: gone where the basic features like limit CPU usage and pause a backup. So Arq ate all my CPU and my fans went crazy and I had no way to control it. This was frustrating.
Also frustrating was to read the developer responses on Twitter. People were upset that features were missing and that the UI was worse and the responses where just a lot of ”The UI will get better and better.”
And when asked about the pause function the response was ”Right. We need to add a pause button. We’ll get on it.” Like, this was a basic feature removed and you didn't even know that until users were complaining? That made me loose a lot of confidence.
The move to Electron was the least of the problems, actually. I don't like it but it's not a big issue for me and the backup engine it self is still native. But then again, I had no issue with the dated look of Arq v5 UI, as long as it was easy to use and understand.
The lack of basic features is a problem though and everything about v6 feels like a early beta version at best. With traces of debugging still left in it.
Did I mention there was no documentation either? At launch it was just 404. Customers asked about it and the reply was “Coming soon”.
And then, in the middle of it all, the Arq Twitter account went away completely with all tweets gone for a couple of days. Not something that restored the trust exactly.
On top of all this I can't access my backups anymore. Hitting the Restore tab only gives me ”Error: unknown path: /api/v1/snapshots”.
This is just one experience but Twitter and Reddit are full of similar reports.
The developer has been working on improvements with multiple releases since the launch and also apologized, but still: the honest surprise is part of the problem. How could he release something like this and then be surprised of the reactions? How could testing not reveal any of the import errors and brokens snapshots that almost everyone seems to have? There were even reports from people who signed up for the short lived beta period that they reported problems and errors before launch, but got no feedback on that.
Oof. Arq has been my primary backup method for years now. I used to use Time Machine for local backups but I never found that it worked as well as Arq for backing up to network shares. I really don't wan't to migrate to an electron app for backups. Maybe it's time to reconsider Time Machine for local backups.
@Ilja There’s been a Windows version for a while now, so I guess Electron enables a lot more code sharing.
@Adrian I had those same issues with importing.
Another issue I forgot to mention: unmounting any drive (even if it’s neither the source nor the destination) will make a running backup abort.
>I really don't wan't to migrate to an electron app for backups.
Only the user interface is in Electron, and it actually looks like pretty much any other Mac app, and mostly feels like one, too. There are some small things missing or off in the UI, but I think overall, the UI is an improvement over older versions.
I haven't looked at the Windows version yet, but presumably, it'll be pretty much the same as the Mac version, which I personally consider a nice bonus.
I agree that this probably shouldn't have shipped at this point. Backup software must be dependable. You can't ship new versions without having documentation ready, or with important features missing. But I also don't think this is a reason to abandon Arq, or go back to something like Time Machine. I'd rather have a mostly Mac-like UI built in Electron than an LSD-trip-like UI built natively.
This is really disappointing. Years ago, I used Arq v1 but eventually gave up because our home internet options were limited to late 1990's level options. Now that we have a functioning broadband connection, I was hoping get back to Arq. This easily preventable disaster has me rethinking that idea.
BTW, if you don't need a nifty UI or even prefer command line tools, check out duplicacy.com - I've been following the development for a few years. It's a x-platform, bare-bones, incremental backup system, optimized for use with remote storage services just like Arq.
And maybe one day there'll be someone helping the developer to write a better UI, even. For now, you at least get a simple Web UI that can do the basic jobs.
So I have to restart with all my backups, i.e., many TBs stored??? It would be helpful if Arq 5 could at least be run in a read-only mode for restores.
@Thomas Tempelmann
I know, not exactly germane to this thread, but thank you again for recommending Acrosync in a prior thread. It gets used every day on my daughter's Surface Pro 3. If Duplicacy works as well as a dedicated cloud backup platform, then maybe I'll check it out too. Thanks!
I too upgraded to Arq 6 right away, having full trust in the developer after relying on Arq 5 for years, then was rudely shocked at what happened. I use Arq for a local backup and a cloud backup to Backblaze B2. I imported the local backup (it failed the first time but I tried again and it seemed to work, and I was able to do a test restore that convinced me it worked). I decided to not import the cloud backup right away, and glad I did. After a couple of weeks, my hourly local backups stopped running for 9 days and never notified me until I rebooted. Upon resuming, the hourly local backup looks like it's going to take many hours to complete (usually it's under 5 minutes). It seems to be fully copying all my files again??? I have no idea what it's doing so I've stopped it, unscheduled it, waiting for a reply for Arq support, and seriously considering switching to something else. I've lost all confidence in Arq.
It's worth knowing that the dev publicly tried to identify one of the complaining beta users on Reddit: https://www.reddit.com/r/Arqbackup/comments/g267zd/i_am_a_big_fan_of_arq_but_arq_6_should_be/fnx1lt2/
Per Arq website:
- We plan to deliver compatibility mode before end of June 2020. (Update July 4th: We need a few more weeks!)
I just downloaded Arc 6, hoping the bugs were fixed, but after opening I noticed the feature to choose which Wifi networks and interfaces to use for backup doesn't exist at all in Are 6, as far as I can tell! That is a crucial feature -- I don't want to be backing up over my cell phone connection. Even if they managed to fix the bugs, this seems to me to be a major loss of functionality.
Arq 5 was quite reliable -- I have no clue what happened with Are 6, did they fire all the original engineers and start over from scratch? I honestly found Arq to be by far the fastest backup solution, very simple and straightforward -- but Are 6 is a bizarre shift. What happened to the team, designers, etc., who built Arq 5?
@Mitsu The Wi-Fi feature is still there, but it moved in the UI because now it can be set per-backup.
