As for space: ZFS can run a transparently compressed storage pool, which in my tests saves about 20% disk capacity. Particularly on a laptop that's key, and it may speed some things up, because generally IO is the bottleneck, not CPU required to decompress, which is why such things as compressed or encrypted swap files are not a problem.

ZFS is the way to go, and should for some reason ZFS be too slow for some media application, then do what most media people do anyway: use a separate drive for recording/streaming, and you can use a legacy file system on that.

Fragmentation actually is quite a problem for some people (again, see the zfs-discuss mailing list -- a great resource if you're interested in learning more about the limited real-world experience here). It isn't for others. I don't know anyone who's tried to stream data from a ZFS file system at high rates, which is where fragmentation tends to be the biggest issue. (Note that Apple’s HFS+ implementation has the ability to preallocate space for a file contiguously on disk, important for audio/video and useful for file copies as well; presumably this feature could also be added to ZFS in the future.)

Since real data, or at least an approximation thereto, is better than guesses, I used rsync to copy most of my laptop disk (with the exception of /Users) to (a) an HFS+ disk image, and (b) a ZFS disk on a Solaris VM. (For those who want to play with ZFS, Solaris 10 U4 runs nicely under VMWare Fusion.) It's fairly large as I have a lot of applications, fonts etc. installed.

HFS+ required 19611576K to store this data (as measured by 'df -k').
ZFS required 19065405K, again as measured by 'df -k'.

I then used 'runat touch filetype' to create a microzap for each file on the ZFS partition.
ZFS required 20243967K after this -- a 6% overhead, or 1.1 GB. Not as bad as I was expecting, actually, but still not cheap.

I did note that Sun lists a suggested project to add additional attributes to each file node proposed by 'a third party vendor'. I don't know whether this was Cluster File Systems (now purchased by Sun) or Apple, but either of them would seem likely candidates. I suspect that project is fairly low on Sun's priority list, but Apple obviously would have an interest in making it happen.

Wouldn't the microzaps be loaded off the same block, resolving them from memory after the 1st?

As for backups, you don't do incrementals forever. Store a full snapshot once in a while (e.g. once a week/month), and store incrementals in between. Unless you're a real data pack-rat, the new snapshots overwrite the old (and the incrementals). That's when your 200GB parallels disk goes away.

Fragmentation: quoting, as I don't have the time to go in-depth on ZFS:

"Currently fragmentation has not been found to be a problem, the general rule with all properly designed filesystems is that you use no more than 90% of the space then you will have no problems with fragmentation. There currently isn't a defragger, if it becomes a problem it will be intergrated into zpool scrub and it will be able to run in the background or in the middle of the night. In the future they will come up recomendations for when and if you should run zpool scrub. It will check all your data for errors and fix any it finds.

I haven't experienced problems with fragmentation and I have exceeded the 90% rule quite frequently almost constantly in fact and have had no problems with fragmentation. I currently have 48 filesystems, and over 300 snapshots. On approximately 100GB of storage."

Lally: If you think ZFS is faster than a normal LVM for large sequential writes (the audio/video case), you should benchmark it. The current implementation breaks both reads and writes into small chunks, even smaller if you're using RAID-Z, which hurts performance substantially. QFS could easily outperform ZFS by 2x or more on identical hardware for streaming read/write workloads. Even UFS was faster for many workloads. (It doesn't help that ZFS fragments your disk to an extreme.)

Scooby/Michael: One issue with the microzap is that it would require that *each* file attribute be given its own 4-byte entry. A FSGetCatalogInfo call would then require about 10 lookups per file. This is expensive in terms of CPU time. (It's only slightly wasteful for space.) Of course, any third-party attributes would still require use of a fatzap, which is 128KB per file (though compression could alleviate this, again at the cost of CPU).

Richard: If your hard disk was actually damaged and portions couldn't be read, you don't need ZFS to find out which files are unreadable. Any utility that scans the disk would do just fine ('find', 'xargs', and 'dd' if you're handy on the command line). ZFS only improves the situation if you have data blocks which contain data which was written incorrectly (or if you use mirroring, which you can also do with Apple RAID, SoftRAID, etc.).

Tom: Snapshots don't really help Time Machine (or similar concepts), because they're not selective. You can't say, I don't want a copy of the 5 GB of pictures I just downloaded (because I'll toss 4 GB of them anyway), but keep the 5 MB of email around. They are an all-or-nothing solution. (And of course, they're not a backup at all, since they share the same media and even the same disk blocks.) Lally's approach would be the right way to use snapshots to advantage in the backup process, but it suffers from the same problem -- you can't ever delete anything. If you accidentally backup your 200 GB Parallels disk, too bad, you've got that 200 GB on your backup disk forever. Real backup software can [usually -- sadly, Retrospect on Mac OS doesn't, though the Windows version does, grrr] let you selectively remove items from the backup.

That said, they are also human, and falable.

MacJournals repeatedly wrote about the impossibility of bringing journaling to HFS+ without major changes that would break everything, and also about how it would be impossible to go to the Intel architecture.

ZFS will come, the only question for now is when.

I wouldn't bet against MacJournals very often, but on this issue, I'll wait to see what surprises Apple has on Leopard-is-Shipping Announcement Day.

They have surprised us before, and will again.

Pete

As for large-scale writing speeds, you'll be using up most of your time spoon-feeding your disks, leaving a bit of spare CPU time for your checksum. Checksumming isn't slow or computationally expensive. It also tells you when your drive's going bad -- nice to know if you don't want to lose your drive altogether when you're editing video.

And, if you decide you want to cut down your write times by having more than one disk store your data (have them write different parts in parallel), ZFS will do it faster than your normal LVM.

Finally, I'm really surprised more people aren't really excited about checkpoints -- it's a dead-simple way to do reliable backups and restores. Keep one checkpoint active after your last backup, and store the diff. Make a new checkpoint for now, and delete the old one. Old data only stays around long enough to be backed up. *And* those backups are serial byte streams, stuff you can shove through gzip before you write it to disk.

Think of all the disk thrashing you don't have to do to make good incremental backups. SuperDuper freaks me out in this area.