Tahoe Free Space Problems

My MacBook Pro went wild last week, once I’ve updated to Tahoe Beta 6. Looks like Spotlight was the culprit, showing high CPU usage and byte writes on `mds_stores` and the entire disk space being eaten.

This is not a new bug, as per your reference article. Users on Sequoia have experienced similar problems. Maybe a regression?

The spotlight problem is at least from Big Sur.

I use Daisy Disk to analyse free space on my hard disk. Usually, it's some TimeMachine snapshots even though I don't use TimeMachine on this computer. Preboot takes 40 GB of free space and an update has another 12 GB. Usually, this is cleared up after restart but this time it didn't help at all.

I don’t think I’ve ever been this reluctant about a macOS release. I’ll definitely slap it on the iPad just to see, but I’m hesitant about the phone and really worried about the Mac.

I think they forget people need these things for work, not just to admire how pretty they are.

Oddly, it cleared up for me without restarting.

The more I read things like this, the more I think APFS was a mistake. Apple is demonstrating that it's beyond their capabilities to do something as seemingly simple as figure out how much disk space is left on your drive, thanks to what I'm gathering are overly complicated features of APFS. At this point I don't see the benefits of it outweighing the downsides and bugs.

I may be off base pointing the finger at APFS specifically, but if ever there was a microcosm that speaks to what's going wrong with macOS, it's this particular ongoing "free space" problem, one that's been getting worse for six major versions.

The Free disk space thing has been a problem for a while. If I remember correctly (maybe I don't...) didn't System Preferences before the redesign to System Settings at least show you the details of what the System Data actually is?

In any case it is unacceptable to tell you half your disk is filled with data you can't touch. Where it is, what it is...well that's none of your business.

Maybe in some cases it is an APFS bug and they can't tell you what the system data is, if they aren't calculating free disk space correctly in the first place. So just remove that UI from System Settings. That'll "fix" that.

@ObjC4Life The weird thing is that my test Mac is a clean system. Pretty much all it has is Xcode. No iCloud Drive, no documents except my Git repo, no photos, etc. I intentionally left a huge amount of free space so I could install OS and Xcode updates. After installing beta 7, it went from showing 40 GB of free space but complaining of being full to showing 132 GB of free space. I have no idea what could account for such a huge difference.

For all APFS's issues, must say I do love the space saving benefits of file clones via copy on write.

I recently also had huge disk space problems. At first an update (pre Tahoe) quietly failed to install. After figured out why, I freed some space, with great difficulty.

Later, macOS complained about the disk being full, which just didn't make any sense to me. After the usual suspects of ~/Library/Caches and friends, as well as ~/Library/Developer/Xcode/ I was still very low, though that helped a little.

A little more digging revealed two main culprits:

1. Simulators
2. Caches in /var/folders/zz

The first was around 300GB (without deleting all of them), the second around 70GB, most of the latter being some sort of diskimaged caches, so you need to restart for the disk space to actually be recovered, the daemon holds onto it.

So I went from 1GB free to 370GB free.

@rajs There's definitely good parts of APFS. Copy-on-write is a good one. And I love being able to have multiple volumes that share space within one container.

I just wish we could have the good parts without the bad parts.

@rajs and @Bri something something ZFS? The Linux of filesystems? Used in enterprise but the somehow unattainable goal of the end user?

Honestly not that familiar with the ins and outs of filesystems but I've heard that one for years and it and nothing like it ever seems to quite come about.

That weird disk space issue is nothing new. I'm also on Sequoia. I have a 1TB SSD that contains less than 600 GB of actual visible data, yet it only shows 100 GB free, with no way to determine the issue or resolve it.

It showed 250 GB free just a week ago.

> @ObjC4Life The weird thing is that my test Mac is a clean system. Pretty much all it has is Xcode. No iCloud Drive, no documents except my Git repo, no photos, etc. I intentionally left a huge amount of free space so I could install OS and Xcode updates. After installing beta 7, it went from showing 40 GB of free space but complaining of being full to showing 132 GB of free space. I have no idea what could account for such a huge difference.

Sounds similar to the issue I was having not too long ago. All I was doing on my Mac was living in Xcode, working on the same project, rebuilding it over and over again. This went on for weeks. I commented about it in your "Spotlight Indexing Running Wild" post. I didn't download anything during that time (or do too much of anything else on my Mac besides working on this one Xcode project) but somehow I ended up with < 1gb of free space and Xcode started failing to build. I suspected that it may of had to do with Time Machine backup being done around that time but maybe not...or possibly there are multiple issues at hand.

I'm on 15.6.1. About to install Tahoe beta in a virtual machine. Therefore I need a bit of free disk space...
I just took a look in System Settings and I have 233 GB of System data. This is quite a big chunk. What it is, I have no idea. Looking..

Xcode Derived Data is pretty huge right now at ~40 GB
Documentation Cache is around ~9 GB
Simulators are ~30 GB..

That's around 80 GB... but the "Developer section" in System Settings is showing only 68GB of "Developer data" so I guess some of this must be getting grouped in the "System Data" section which seems wrong. I consider "System data" to be macOS files you aren't allowed to remove b/c it is important to the system. Developer caches/simulators are my files which I am free to manage. Caches != system data IMO. Not sure how they are breaking all this down in System Settings....

Just getting rid of all the developer data I described above (I can't get rid of it all...I need some simulators but I can chuck some) that would free up around 80 GB. Assuming all of this "Developer Data" is in the "System Data" group and I got rid of it all:

233 GB - 80 GB = 153GB. What the hell is making up the other 153GB of "System Data?" And if all this dev data is being computed as "System Data" then what is the 68GB of "Developer data"? I wonder if some of this stuff is being double counted and put in both groups....

Now looking in Disk utility.... looks like I got 40 GBs in local Time Machine snapshots too. Grr..

So I deleted a simulator disk image in Finder. Freed like 20gb of "system data" (not developer data). So that gets computed as "System Data".

Then....I deleted some "iOS device Support" entries in System Settings (what is shown to you in the System Settings UI) and the amount of "Developer data" in the graph when down to 27GB. Progress..

Then... I went in Xcode Settings -> Platform Support and removed a bunch of old entries. Got rid of iOS 15, iOS 18.2 etc. According to Xcode all these entries were like many GBs on disk..

And guess what? After doing all that System Settings is now showing that "System Data" is now 260 GB 😳. I started with 233 GB of System Data. All I did was delete stuff and the amount of system data *grew* by nearly 30 GB.
Feels like they want to tell me my disk is almost full no matter what.

The question of free space is complicated because there are multiple ways in which it is accounted for. There is the information in disk utility, the info in Finder & Get Info and then the storage summary in System Settings.

Disk utility's accounting is complicated by the fact that volumes can share free space. The exact meaning of "purgeable" is another complication.

Finder has to contend with the fact that, thanks to COW, allocations may need to be accounted for under multiple subdirectories. There is also the fact that some files are only proxies (not sure the official term) for files stored remotely (via FileProvider). Local files in FileProvider directory trees may also be added up under a subdirectory containing a backing store, even though they are only thin files.

Get Info provides some delineation between actual allocations vs proxies, thin files and COW.

System Settings applies its own poorly explained schema. The summary of the "Applications" category provides a single number that includes more than just the footprint of the application bundle in the */Applications directories. This number is repeated at the top of the detail view, but the list of individual applications only shows the footprint of the Application bundle. What else is included in the top-line #? If anyone knows, they aren't saying.

The System Data category in System Settings is another black box. We only get a top line # with no explanation about what's included. With a little investigation it's clear that must be spread across multiple subdirectory trees. Does it include purgeable data? Who knows?

I'm convinced that no one person at Apple understands how all the values in System Settings are calculated, but maybe someone does and they just DGAF about making it clear.

@eas I think COW only affects calculations of used space, not free space.

One possible reason for disk usage is, as mentioned here previously, local snapshots (for Time Machine).

My Mac mini has multiple VM's running all day. Local snapshots of the VMs' data caused my Mac to run out of space. This even caused VM's to complain that they couldn't save data anymore.

To solve this I created a small LaunchDaemon that deletes all excess snapshots and leaves just four of them, for Time Machine and to have a fallback after accidentally deleting a file. It runs without root privileges under any admin account. I'll try to paste it here but I'm not sure if will get scrambled, so beware.

#!/bin/sh
# Find list of snapshots, skipping the first four
excessshots=`/usr/bin/tmutil listlocalsnapshotdates | sed "1d" | sort -rn | sed "1,4d"`
# Delete excess snapshots
if [ -n "$excessshots" ]; then
  for i in $excessshots; do
    /usr/bin/tmutil deletelocalsnapshots $i
  done
fi