{"id":37794,"date":"2022-12-01T16:45:45","date_gmt":"2022-12-01T21:45:45","guid":{"rendered":"https:\/\/mjtsai.com\/blog\/?p=37794"},"modified":"2022-12-02T15:48:46","modified_gmt":"2022-12-02T20:48:46","slug":"long-app-hangs-due-to-spotlight","status":"publish","type":"post","link":"https:\/\/mjtsai.com\/blog\/2022\/12\/01\/long-app-hangs-due-to-spotlight\/","title":{"rendered":"Long App Hangs Due to Spotlight"},"content":{"rendered":"<p>Whenever I mount a hard drive with lots of files, typically a clone drive, various other apps that deal with files <a href=\"https:\/\/twitter.com\/mjtsai\/status\/1590060315366068224\">often hang<\/a>. An individual hang can last for minutes to over half an hour. If the app is doing a long series of file operations, which would normally take fractions of a second, it can be unusable until some time after the drive has been unmounted. This happens even though the apps in question are not accessing files on the hard drive. The problem has been occurring since Monterey but has gotten a lot worse in Ventura. It got to the point where I didn&rsquo;t want to do any backups during the day since my Mac could essentially become unusable for hours, even after pausing or aborting the backup.<\/p>\n\n<p>Sampling the hung apps shows that they are waiting (for an XPC process) to read or write Spotlight metadata. I don&rsquo;t know exactly what&rsquo;s going on, but it&rsquo;s <em>as if<\/em> the newly mounted drive generates lots of potential work for Spotlight to update its index. You would think that each volume would have its own work queue, but it seems like there&rsquo;s a single queue so that operations for the slow hard drive, which are not time-sensitive, block high-priority operations for the internal SSD. Maybe this is exacerbated by the fact that my clone drives get ejected after the backup completes so that the Spotlight index is always way out of date.<\/p>\n\n<p>Once I traced the problem to Spotlight, the obvious workaround was to exclude those volumes from indexing. This is, in fact, effective, though it has some problems:<\/p>\n<ul>\n<li>Sometimes I do want to use Spotlight with these volumes.<\/li>\n<li>APFS Time Machine volumes cannot be excluded from Spotlight, though fortunately they seem to cause fewer problems because they are plugged in for long periods of time and so tend to have up-to-date indexes.<\/li>\n<li>I encountered lots of cases where I couldn&rsquo;t exclude the volume from Spotlight:\n\t<ul>\n\t<li>Sometimes it takes half an hour or so for the drive to finish mounting, yet before that happens Spotlight has already bogged down.<\/li>\n\t<li>Sometimes nothing happens when I click the <strong>Spotlight Privacy&#8230;<\/strong> button in System Settings.<\/li>\n\t<li>Sometimes the privacy list opens but is incorrectly shown as empty, and nothing happens when I drag into it.<\/li>\n\t<li>Sometimes I get an error when dragging a volume into the privacy list.<\/li>\n\t<li>Sometimes, when trying to exclude via <code>mdutil<\/code>, it fails due to &ldquo;unknown indexing state&rdquo; or &ldquo;Spotlight server is disabled.&rdquo;<\/li>\n\t<\/ul>\n<\/li>\n<\/ul>\n<p>Some drives were stubborn, and I had better luck excluding them from Spotlight using a Mac running macOS 10.14. I hope to eventually get all of my backup drives excluded, at which point I expect the hangs to stop completely.<\/p>\n\n<p>Previously:<\/p>\n<ul>\n<li><a href=\"https:\/\/mjtsai.com\/blog\/2022\/08\/17\/nightly-time-machine\/\">Nightly Time Machine<\/a><\/li>\n<li><a href=\"https:\/\/mjtsai.com\/blog\/2022\/08\/16\/removing-unwanted-apfs-time-machine-backups\/\">Removing Unwanted APFS Time Machine Backups<\/a><\/li>\n<li><a href=\"https:\/\/mjtsai.com\/blog\/2022\/07\/18\/invasive-spotlight-indexing\/\">Invasive Spotlight Indexing<\/a><\/li>\n<\/ul>\n\n<p id=\"long-app-hangs-due-to-spotlight-update-2022-12-02\">Update (2022-12-02): <a href=\"https:\/\/twitter.com\/nicolaih\/status\/1598684604675854337\">Nicolai Henriksen<\/a>:<\/p>\n<blockquote cite=\"https:\/\/twitter.com\/nicolaih\/status\/1598684604675854337\">\n<p>Aha! Thats why! I have a TimeMachine backup disk that makes my entire machine halt completely whenever it is attached. It does not return to normal even when connected overnight. The TM disk was only connected once a week - until I gave up.<\/p>\n<\/blockquote>","protected":false},"excerpt":{"rendered":"<p>Whenever I mount a hard drive with lots of files, typically a clone drive, various other apps that deal with files often hang. An individual hang can last for minutes to over half an hour. If the app is doing a long series of file operations, which would normally take fractions of a second, it [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"apple_news_api_created_at":"2022-12-01T21:45:48Z","apple_news_api_id":"1948530c-730c-4d04-9aa0-bc85261eb335","apple_news_api_modified_at":"2022-12-02T20:48:48Z","apple_news_api_revision":"AAAAAAAAAAAAAAAAAAAAAw==","apple_news_api_share_url":"https:\/\/apple.news\/AGUhTDHMMTQSaoLyFJh6zNQ","apple_news_coverimage":0,"apple_news_coverimage_caption":"","apple_news_is_hidden":false,"apple_news_is_paid":false,"apple_news_is_preview":false,"apple_news_is_sponsored":false,"apple_news_maturity_rating":"","apple_news_metadata":"\"\"","apple_news_pullquote":"","apple_news_pullquote_position":"","apple_news_slug":"","apple_news_sections":"\"\"","apple_news_suppress_video_url":false,"apple_news_use_image_component":false,"footnotes":""},"categories":[2],"tags":[146,131,30,2077,2223,1042,174,1181,1473],"class_list":["post-37794","post","type-post","status-publish","format-standard","hentry","category-technology","tag-backup","tag-bug","tag-mac","tag-macos-12","tag-macos-13-ventura","tag-spotlight","tag-storage","tag-system-preferences","tag-xpc"],"apple_news_notices":[],"_links":{"self":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/37794","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/comments?post=37794"}],"version-history":[{"count":4,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/37794\/revisions"}],"predecessor-version":[{"id":37808,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/37794\/revisions\/37808"}],"wp:attachment":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/media?parent=37794"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/categories?post=37794"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/tags?post=37794"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}