{"id":25067,"date":"2019-04-22T14:56:41","date_gmt":"2019-04-22T18:56:41","guid":{"rendered":"https:\/\/mjtsai.com\/blog\/?p=25067"},"modified":"2020-12-31T16:42:42","modified_gmt":"2020-12-31T21:42:42","slug":"performance-considerations-when-reading-directories-on-macos","status":"publish","type":"post","link":"https:\/\/mjtsai.com\/blog\/2019\/04\/22\/performance-considerations-when-reading-directories-on-macos\/","title":{"rendered":"Performance Considerations When Reading Directories on macOS"},"content":{"rendered":"<p><a href=\"https:\/\/blog.tempel.org\/2019\/04\/dir-read-performance.html\">Thomas Tempelmann<\/a>:<\/p>\n<blockquote cite=\"https:\/\/blog.tempel.org\/2019\/04\/dir-read-performance.html\"><p>Unfortunately, with Apple&rsquo;s new file system APFS, and the fact that any macOS running High Sierra or Mojave got their startup volume converted from HFS+ to APFS, search performance has decreased by factor 5 to 6!<\/p><p>[&#8230;]<\/p><p>I&rsquo;ve tried to find out which of the various methods of reading directories, looking only for file names, is the fastest: I had to scan the same directory tree with every method separately.<\/p><p>[&#8230;]<\/p><p><code>contentsOfDirectoryAtURL <\/code>and <code>getattrlistbulk<\/code> perform equally indeed, just as predicted, with the latter usually being a bit faster once the data comes from the cache.<\/p><p>On APFS, NTFS and SMB, <code>opendir()<\/code> is significantly faster than the other methods, which is quite surprising to me.<\/p><p>[&#8230;]<\/p><p>When accessing a Mac via SMB, <code>contentsOfDirectoryAtURL<\/code> is faster than the other methods, but only on the first run (see red field). Once the caches have been filled, it&rsquo;s slower. I can&rsquo;t make sense of it, but it&rsquo;s a very consistent effect in my tests.<\/p><\/blockquote>\n\n<p>Previously: <a href=\"https:\/\/mjtsai.com\/blog\/2017\/07\/20\/apfs-and-fast-catalog-search\/\">APFS and Fast Catalog Search<\/a>.<\/p>\n\n<p id=\"performance-considerations-when-reading-directories-on-macos-update-2019-04-23\">Update (2019-04-23): <a href=\"https:\/\/blog.tempel.org\/2019\/04\/dir-read-performance.html\">Thomas Tempelmann<\/a>:<\/p>\n<blockquote cite=\"https:\/\/blog.tempel.org\/2019\/04\/dir-read-performance.html\">\n<p><code>fts_open()<\/code>&nbsp;\/&nbsp;<code>fts_read()<\/code>&nbsp;are, in most cases, faster than <code>readdir()<\/code>,&nbsp;<code>contentsOfDirectoryAtURL<\/code>&nbsp;and&nbsp;<code>getattrlistbulk<\/code>. Exceptions are network protocols, where especially the retrieval of additional attributes makes it slower than the other<\/p>\n<\/blockquote>","protected":false},"excerpt":{"rendered":"<p>Thomas Tempelmann: Unfortunately, with Apple&rsquo;s new file system APFS, and the fact that any macOS running High Sierra or Mojave got their startup volume converted from HFS+ to APFS, search performance has decreased by factor 5 to 6![&#8230;]I&rsquo;ve tried to find out which of the various methods of reading directories, looking only for file names, [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"apple_news_api_created_at":"2019-04-22T18:56:44Z","apple_news_api_id":"cf6f9710-ef68-4fec-ab23-8dfdb5cb31a0","apple_news_api_modified_at":"2020-12-31T21:42:45Z","apple_news_api_revision":"AAAAAAAAAAAAAAAAAAAAAQ==","apple_news_api_share_url":"https:\/\/apple.news\/Az2-XEO9oT-yrI439tcsxoA","apple_news_coverimage":0,"apple_news_coverimage_caption":"","apple_news_is_hidden":false,"apple_news_is_paid":false,"apple_news_is_preview":false,"apple_news_is_sponsored":false,"apple_news_maturity_rating":"","apple_news_metadata":"\"\"","apple_news_pullquote":"","apple_news_pullquote_position":"","apple_news_slug":"","apple_news_sections":"\"\"","apple_news_suppress_video_url":false,"apple_news_use_image_component":false,"footnotes":""},"categories":[2],"tags":[1395,537,30,1609,138,2029],"class_list":["post-25067","post","type-post","status-publish","format-standard","hentry","category-technology","tag-apple-file-system-apfs","tag-filesystem","tag-mac","tag-macos-10-14","tag-optimization","tag-server-message-block-smb"],"apple_news_notices":[],"_links":{"self":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/25067","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/comments?post=25067"}],"version-history":[{"count":2,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/25067\/revisions"}],"predecessor-version":[{"id":25079,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/25067\/revisions\/25079"}],"wp:attachment":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/media?parent=25067"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/categories?post=25067"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/tags?post=25067"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}