Quantum Computing and APFS: Free and Used Space
In APFS, snapshots are made as part of Time Machine backups, on some occasions such as prior to installing a macOS update, and when the user initiates them. What happens when an APFS snapshot is made is that a complete copy is made of the file system metadata, which is very quick indeed and doesn’t involve the copying of any other file data.
However, to preserve all the files at the moment that the snapshot is made, as those files subsequently change, their original data are retained so long as the snapshot is kept. Let’s say that, in one snapshot, there’s a certain file of 1 GB in size, which then changes completely so the whole 1 GB is rewritten. So long as that snapshot is retained, its original 1 GB of data is retained, as well as its new 1 GB. So although the snapshot itself doesn’t take up much space, it stops a lot of old data from being freed up for reuse.
Time Machine purges old snapshots automatically, but by default retains the last 24 hours of hourly snapshots, which will take a total space similar to the amount of data backed up over that period. In my case, that’s typically around 30 GB at any time, but if you manipulate large media files, or old snapshots aren’t purged properly, it could easily require hundreds of GB.
I find the whole idea of free space confusing with Mojave and APFS. Last week, after updating Xcode, I started getting warnings that my MacBook Air’s SSD was almost full. I deleted about 20 GB of files but saw little improvement in the amount of available space reported by Finder. And I continued to get the warnings. As far as I could tell, based on reports from OmniDiskSweeper and trying to figure out which files had been deleted and added, I should have netted about 14 GB more free space than I started with, yet Finder showed about 6 GB less.
My first thought was that the culprit was snapshots, but Disk Utility showed no purgeable space, and tmutil listlocalsnapshots showed no snapshots. I guess this indicates that the disk really is almost full, but I can’t figure out why.
My current theory is that the problem is due to a 20 GB proxies folder in Photos’s library (which is strangely not included in Manage Storage’s report). I don’t have a measurement of how large this was before, but I suspect it used to be much smaller. It’s not clear how I might compact it or why Photos isn’t automatically deleting old data now that it can tell that the disk is almost full. I will probably end up deleting the Photos library and syncing a new one down from iCloud. But that’s only a potential solution because this Mac doesn’t contain any Photos projects; those are not stored in the cloud and so would be lost if I were to attempt this.
Previously:
- How Time Machine Makes Backups With APFS
- The Definitive Guide to iCloud Photos in 2019
- How OmniDiskSweeper Reports Free Space
- Photos Needs Better Storage Management
- Local Time Machine Uses APFS Snapshots
Update (2019-04-11): My iMac does show Time Machine snapshots, yet Disk Utility still says there’s zero purgeable space.
Update (2019-04-23): Tricertops (via Peter Steinberger):
I have 40 GB of free space, but Xcode cannot be updated. Maybe the next macOS will finally fix things? Or the one after that?
I just had the same problem updating Xcode on a Mac that purportedly had 30 GB of free space, with the expanded .xip file needing less than 10 GB. After waiting a while, the displayed amount of free space didn’t change, but expanding the .xip no longer reported a disk space error.
Update (2019-04-28): Matt Birchler:
iPad: You have about 70GB of free space
Me: I’d like to download this 2GB file from iCloud
iPad: I don’t think so, chief
Me: Ok, I’ll remove some stuff manually, can I see what files are taking up space?
iPad: What sort of agreement do you think we have?
Update (2019-05-09): Lloyd Chambers:
The purgeable space concept on APFS is mystifying.
Update (2019-07-29): I recently encountered a problem where I couldn’t back up via SuperDuper because the destination drive had 250 GB free, but it was all purgeable. Both tmutil and diskutil reported no space consumed by APFS snapshots, so it’s not clear how I could ever purge that space. I ended up having to reformat the drive.
Update (2019-09-17): Tom Harrington:
I have 78GB free on this Mac according to Finder but I can’t decompress Xcode because there’s not enough space. I know it’s APFS stuff and can fix, but how do non geeks deal with this kind of thing?
Update (2019-12-20): Oluseyi Sonaiya:
32 GB free.
Update Xcode to 11.1
20 minutes later: “This application can not be installed. Insufficient disk space.”
20 GB free.
It’s insufficient because YOOOOOOUUUU took up 12 GB?!
6 Comments RSS · Twitter
Try looking in Disk Utility in the repair section for Macintosh HD and see what you have for snapshots.
You could check what Daisy Disk ( https://daisydiskapp.com ) reports. And/or CleanMyMac X ( https://macpaw.com/cleanmymac ). They might be able to identify the culprit, or at least tell you the folder that's larger than you'd expect it to be. E.g. I frequently have the issue of Time Machine needing 30GB spare disk when doing the approximately monthly complete comparison with the backup, but when the remaining local space it too low, it simply stalls/quits without removing the index/cache it created locally, and without telling you that backups won't run properly, until you create more free disk space.
It seems there are now two distinct counts: “available” disk space and “free” disk space. When I enabled “Optimize Mac Storage” under Photos my available space increased, as did the share of that available space which was listed as “purgeable”, but my free disk space was unchanged.
@Ghost Do the Available and Purgeable amounts in Finder’s Get Info match what you see in Disk Utility? For me, they don’t.
Shameless plug here, but I think it's pretty relevant. Carbon Copy Cloner will list all of the snapshots on your volumes. It also shows their individual sizes and total size, and you can even delete them (more easily than with tmutil). https://www.youtube.com/watch?v=4wqAC4YXiaY
