Spotlight Indexing Running Wild

I found where it just stops showing applications. It's as if the index breaks. I don't understand searching for application is one of those common things, how can this even break?

My understanding is that some Time Machine processes at least require Spotlight to be enabled on the internal drive, meaning that users who disable Spotlight wholesale might get more than they bargained for.

Old notes of mine state that disabling Spotlight on the internal drive while using Time Machine may result in Spotlight being forced back on by macOS for its own purposes, but that may be complete nonsense.

Does anyone have any insights on how Time Machine and Spotlight interact?

I had this happen to me on my M3 Max MacBook Pro.

I'd leave the machine to fall asleep overnight, come back the next day and the thing is boiling in its own heat. Activity Monitor showed that the corespotlightd process was writing TERRABYTES of data to the SSD. Spotlight performance was also slow.

Unfortunately, that iboysoft article won't actually solve the problem, at least not in my experience. The real fix is in this reddit thread: https://www.reddit.com/r/MacOSBeta/comments/1dfo2sl/ridiculously_high_disk_write_rate_from_unknown/

This was the only thing that worked for me, but I hated having SIP disabled, especially on a portable computer. Ultimately I decided I had to wipe the laptop to save it from chewing itself to pieces. Especially since you can't swap out the SSD inside these machines. I'm lucky I bought a machine config such a powerful/big SSD that was probably able to wear-level the damage, but not wiping the laptop would have been a huge, huge risk.

@Ben To be clear, excluding your drive from Spotlight didn’t work, and the fix was to turn off SIP (csrutil disable) and then use sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.metadata.mds.plist?

I do time machine backups on a NAS. Pretty sure something related to Time Machine causes system data to explode. Or at least macOS says so. Not sure if it’s related to this but i’ve been having this problem I think since ventura

Just working in Xcode for weeks on the same project and not downloading much of anything then all of a sudden i have like 2mb of free disk space..just suddenly out of nowhere and i get a warning about it. System settings has “system data” eating over 50% of my drive. Everytime. system data balloons. In their infinite wisdoms system Settings completely hides what the “system data is”…wtf are they doing filling 50% of the disk with shit and not reporting where the files are..

I was doing hourly Time Machine backups and it was happening constantly so i switched to weekly backups and that helped…problem always seems to occur shortly after a time machine backup, happened today…

My Xcode project wouldn’t build suddenly with a “code signing error” i think i was out of disk space. Cleared derived data and i suddenly got 100 gb back. I don’t know where they are getting these numbers from.

err i’m
backing up to an external disk so you don’t fill the internal one

@MJTSAI - Yep, that's correct!

I tried excluding my internal disk from Spotlight, tried deleting the index and restarting, tried turning spotlight off using mdutil... That crazy process kept coming back and writing TB of data. Disabling SIP and then running that launchctl command to disable process from ever starting was the only thing that actually worked on my machine.

I think I first noticed the problem in 15.3. After 15.4 came out, I re-enabled spotlight by undoing that launchctl command to see if that update fixed it, but it again started writing tons of data to disk almost immediately on system start.

At first I thought maybe the index had gotten corrupted, but since deleting the index didn't fix, it's lead me to think that something else is the culprit, but I haven't figured out what (node_modules, flutter framework files, Xcode libs...?). I've been vigilant about it ever since, occasionally checking the disk usage tab in activity monitor to make sure it hasn't gone on the fritz again.

@ObjC4Life - I started noticing that system data/Time Machine problem several years ago, when I was using my Mac mini (2018) as my daily driver. I eventually ended up disabling Time Machine and made a little Shortcut action to rsync stuff to an external disk. I added the shortcut to the menu bar and just ran it whenever I thought about it.

A lot of my working files are either versioned under git, or they're in iCloud Drive or on my FreeBSD server, so not having it automated didn't really bother me.

@Ben Yea I might have to do something like that. I don't want to stop using Time Machine but I'm getting fed up. Thanks for sharing. I never filed a bug (because they would ask for a sysdiagnose). Figured I couldn't be the only one!

As I mentioned lowering the Time Machine backup frequency seems to cause the issue to happen less often, but it still happens. Not sure if Spotlight is related in my case or not.

Never could find any large files when drilling through my disk in Finder List View with "Calculate View sizes" on. Not sure where they are hiding that over 250 GB of system data at!

A long time ago I believe manually deleting Time Machine local snapshots helped but then the System would just fill her up again in no time. More recently I think I tried deleting local snapshots and it didn't work. Kind of got exhausted by it all and gave up because I was busy. At the very least System Settings should show you where the System files are in the UI even if it didn't let you delete them from the UI (to prevent users from hurting themselves). You could at least file a more informative bug report, more easily.

Instead they show you files for *everything else* but when it comes to "System data" it's "tough shit kid."