Archive for April 28, 2025

Monday, April 28, 2025

Revisiting ZFS for Mac

Hacker News is highlighting Adam Leventhal’s 2016 post (2016 comments) about Apple’s Leopard-era support for ZFS:

A few weeks before WWDC 2007 nerds like me started to lose their minds: Apple really was going to port ZFS to Mac OS X.
[…]
ZFS was to bring to Mac OS X data integrity, compression, checksums, redundancy, snapshots, etc, etc etc. But while energizing Mac/ZFS fans, Sun CEO, Jonathan Schwartz, had clumsily disrupted the momentum that ZFS had been gathering in Apple’s walled garden. Apple had been working on a port of ZFS to Mac OS X. They were planning on mentioning it at the upcoming WWDC. Jonathan, brought into the loop either out of courtesy or legal necessity, violated the cardinal rule of the Steve Jobs-era Apple. Only one person at Steve Job’s company announces new products: Steve Jobs. “In fact, this week you’ll see that Apple is announcing at their Worldwide Developer Conference that ZFS has become the file system in Mac OS 10,” mused Jonathan at a press event, apparently to bolster Sun’s own credibility.
Less than a week later, Apple spoke about ZFS only when it became clear that a port was indeed present in a developer version of Leopard albeit in a nascent form. Yes, ZFS would be there, sort of, but it would be read-only and no one should get their hopes up.
[…]
By the time Snow Leopard shipped only a careful examination of the Apple web site would turn up the odd reference to ZFS left unscrubbed. Whatever momentum ZFS had enjoyed within the Mac OS X product team was gone.

Here’s what I wrote in 2007. Revisiting some of the issues:

With Time Machine also being announced with Leopard, there was initial speculation that it was implemented using ZFS snapshots. Of course, that turned out not to be the case. Apple did eventually reimplement it using APFS snapshots.
ZFS was incompatible with HFS+ in that it didn’t support full-length Unicode filenames. It now supports filenames up to 1,023 bytes, which is more than APFS. So this is surely something Apple could have added if it wanted. (I’m not actually sure what the APFS limit is. Apple’s documentation doesn’t seem to say. Wikipedia says 255 UTF-8 characters, which I’m pretty sure is wrong because I think it supports at least 255 UTF-16 characters like HFS+. I was unable to set more using Finder. In any case, even 255 full Unicode characters encoded as UTF-8 is less than 1,023 bytes.)
ZFS was incompatible with Mac software in that it only supported bag-of-bytes filenames, not case-insensitivity or Unicode normalization. Apple was never fully consistent here, in that at the time it did support Mac OS X on UFS (which lacked both) and HFSX (which was case-sensitive). There was speculation that Apple could address this using higher level APIs. This is the path it took with the initial APFS release, which was no better than ZFS in this regard. By 2017, Apple had announced a new version of APFS that did support Unicode normalization.
I wrote that, unless Apple was already very far along in developing its own new file system, it would choose to get a head start by using a modified version of ZFS to address the above issues. This turned out to be totally wrong, as Apple instead scrubbed nearly all mentions of ZFS. APFS was not even available in developer beta until 2016, at which point it had been in development for only 2 years. It remains somewhat of a mystery what Apple was doing in the interim, aside from working on Core Storage.
ZFS had inefficient storage for extended attributes. I didn’t think this was a big deal at the time because Apple was barely using them. Since then, Apple has been using them a lot more, and APFS offers optimized storage for smaller xattrs.
It wasn’t until 2018 that Apple documented APFS, with the encryption documentation not coming until 2019.
Fast directory sizing was announced with the initial version of APFS but still hasn’t shipped in a form that works with Finder.
There are still very few tools available for working with APFS. This might have been better with ZFS.
I still wish that Apple had supported data integrity features like ZFS and think it’s bizarre that they suggested this was not needed due to their superior hardware. With APFS, I have continued to see file corruption, drives that mount but don’t work properly, and drives that suddenly don’t mount at all. In theory, ZFS could have offered more robustness or at least earlier detection.
Transparent compression would have been nice, too.
APFS’s awful performance on spinning hard drives continues to be an issue, with even small folders sometimes taking more than a minute to display in Finder. ZFS also uses copy-on-write, but it was designed before flash, so I wonder whether it would have worked better in this regard (e.g. keeping the metadata contiguous).
It’s possible that ZFS wouldn’t have worked with iOS due to its memory requirements. I’ve also read that this was solvable but don’t really know the details. And, of course, iPhones have a lot more RAM now.

Previously:

Apple File System (APFS) Extended Attributes File System History Legal Mac Mac OS X 10.5 Leopard macOS 15 Sequoia Open Source Steve Jobs Storage Sun Microsystems Time Machine Unicode ZFS

3 Comments

Synology Hard Drive Locking

Patrick Kennedy (via Hacker News):

According to HardwareLuxx, Synology is on a rough course with generations-old sub-par NAS hardware and now appears to be locking its NAS units to its own branded hard drives in its upcoming 2025 Plus models. This is a shame since a few years ago, Synology had neat hardware.
[…]
Without scale, and a library of patents, it is a very hard market to enter. As a result, Synology must be simply re-branding drives. Labeling drives as a “Dell”, “NetApp”, “HPE”, or other big vendor drive has been going on for years (decades?) on both the hard drive and SSD sides of storage.
[…]
Let us just call this what it is. It is a grab for extra margin dollars. The challenge is that it is bad for Synology’s customers. For example, the Synology Plus series only scales to 16TB currently with the HAT3310-16T. Synology’s enterprise series scales to 20TB. WD Red Pro drives are already pushing 26TB.
[…]
When a drive fails, one of the key factors in data security is how fast an array can be rebuilt into a healthy status. Of course, Amazon is just one vendor, but they have the distribution to do same-day and early morning overnight parts to a large portion of the US.
[…]
Additionally, there can also be concerns about drive availability in the long-term. If your NAS is vendor-locked to only use Synology drives, then as owner of that NAS you are fully dependent upon Synology’s survival as a company and that they would continue manufacturing drives in the capacity points that you want.

Kevin Purdy:

Popular NAS-maker Synology has confirmed and slightly clarified a policy that appeared on its German website earlier this week: Its “Plus” tier of devices, starting with the 2025 series, will require Synology-branded hard drives for full compatibility, at least at first.
“Synology-branded drives will be needed for use in the newly announced Plus series, with plans to update the Product Compatibility List as additional drives can be thoroughly vetted in Synology systems,” a Synology representative told Ars by email. “Extensive internal testing has shown that drives that follow a rigorous validation process when paired with Synology systems are at less risk of drive failure and ongoing compatibility issues.”
[…]
As previously noted by the German press release, Synology Plus models purchased prior to the 2025 series will continue to support third-party drives at their current level.

Rui Carmo:

I’ve been a very happy Synology customer for over 15 years, but this piece of news (which echoes the Western Digital fracas from a couple of years ago) makes me wonder if I will keep recommending them.

Casey Liss:

I think my appreciation for a product crosses the rubicon into love when it regularly and repeatedly demonstrates one trait: respect for the user.

[…]

I’m not sure if I was more of a fan of Synology or Sonos, but suffice it to say, I was a superfan of both. I just replaced my original Synology last year, and I’m sad to say that the one I just got is likely to be my last.

He kind of walks this back after discussion on Accidental Tech Podcast, where it’s suggested that he should just consider the extra cost per drive as part of the total price. He has no problem paying a bit more for other premium products that he loves, so why should this be different? But I think it is different because:

At minimum, it seems like Synology is trying to hide the price increase rather than being forthright.
It seems like the purported benefits of Synology-branded drives are either very minimal or entirely fake. There are no specifics given, and existing third-party drives are still supported. Is this just FUD?
Money aside, the branded drives may not be available in the same capacities or with the same shipping speed. If you keep a bunch of blank drives on hand for other purposes, or if you want to repurpose an existing drive for use in the Synology, that will no longer be possible.

Eric Schwarz:

While I usually enjoy the discussion on the show, it felt a bit dismissive that his concerns were met with the sentiment that these companies have moved on to a different target audience that isn’t him. While that may be true in some instances, the examples he cites (Eero, Sonos, and Synology) aren’t really making big shifts.

[…]

I found Synology’s move away from third-party drive support equally disheartening as this feels like a “fix” for a problem most people never encountered. Even if Synology is going more towards the enterprise market, no one is cross-shopping a 2-bay compact NAS with a massive rack-mount unit. In my time in higher education IT where every dollar matters, I’ve also found that the various DS-21X models were great for one-off installations and we could use any of the extra drives we had on-hand (I ran a personal DS-216+II with a run-of-the-mill Seagate drive I pulled from a dead enclosure for way longer than I probably should’ve, but it was fine). While there might be some instances where having fully-supported equipment makes sense, hard drives are such an established technology that the concerns of compatibility feel a bit manufactured. At the very least, offer an advanced mode to run things in an unsupported manner for enthusiasts.