Tuesday, March 30, 2021

Sparse Files Are Common in APFS

Howard Oakley:

Increasing numbers of files written by all sorts of different apps and services consist of large voids, between islands of meaningful data. Storing lots of void data is wasteful, so what APFS tries to do is store only the real data.


Sparse files are kept in sparse format as much as possible, and when copied or duplicated within the same volume should be kept in sparse format. Copying them between different volumes and disks isn’t so predictable, and sometimes leads to them ‘exploding’ to full size. That is normal when they’re copied to file systems like HFS+ which don’t support sparse files, and to iCloud.

You should expect all sparse files to be expanded fully when they’re backed up to HFS+ disks, as with Time Machine prior to Big Sur, which may not estimate their expanded size correctly either, as I have described. Expansion takes place at the source of a copy: for example, if you copy a sparse file from your internal APFS disk to an external disk in HFS+ format, the full expanded size of data will have to be copied across to the external disk.


Throughout my quest for these elusive sparse files, I had assumed that only certain apps could create them. That isn’t true: macOS now defaults to creating all files in sparse format when certain conditions are met.


When you use the FileHandle class to create a new write handle, a sparse file is created automatically. For example, if you write a block of data, then seek one block by calling seek(toFileOffset:), and then write another block, the data stored on disk is organized as follows:

Comments RSS · Twitter

Leave a Comment