{"id":29366,"date":"2020-06-26T16:16:31","date_gmt":"2020-06-26T20:16:31","guid":{"rendered":"https:\/\/mjtsai.com\/blog\/?p=29366"},"modified":"2025-07-08T21:56:52","modified_gmt":"2025-07-09T01:56:52","slug":"reverse-engineering-macos-11-0","status":"publish","type":"post","link":"https:\/\/mjtsai.com\/blog\/2020\/06\/26\/reverse-engineering-macos-11-0\/","title":{"rendered":"Reverse Engineering macOS 11.0"},"content":{"rendered":"<p><a href=\"https:\/\/developer.apple.com\/documentation\/macos-release-notes\/macos-big-sur-11-beta-release-notes\">Apple<\/a>:<\/p>\n<blockquote cite=\"https:\/\/developer.apple.com\/documentation\/macos-release-notes\/macos-big-sur-11-beta-release-notes\"><p>New in macOS Big Sur 11 beta, the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail. Instead, check for library presence by attempting to <code>dlopen()<\/code> the path, which will correctly check for the library in the cache.<\/p><\/blockquote>\n\n<p><a href=\"https:\/\/twitter.com\/pedantcoder\/status\/1275453028305264640\">Pierre Habouzit<\/a>:<\/p>\n<blockquote cite=\"https:\/\/twitter.com\/pedantcoder\/status\/1275453028305264640\"><p>The only impact is if you are doing runtime detection\/search of library by path yourself. Which is a terrible idea for perf anyway.<\/p>\n<p>iOS has been like that for a decade already.<\/p>\n<\/blockquote>\n\n<p>The goal was optimization, but unfortunately it does make reverse engineering more difficult.<\/p>\n\n<p><a href=\"https:\/\/twitter.com\/jckarter\/status\/1275456473703706633\">Joe Groff<\/a>:<\/p>\n<blockquote cite=\"https:\/\/twitter.com\/jckarter\/status\/1275456473703706633\">\n<p>The shared cache isn&rsquo;t encrypted or anything, and dyld is in the Darwin source dumps. The shared cache format may not be stable, but isn&rsquo;t secret either<\/p>\n<\/blockquote>\n\n<p>The data is there, but there currently aren&rsquo;t tools that can get it into a useful format like we had before.<\/p>\n\n<p><a href=\"https:\/\/twitter.com\/stroughtonsmith\/status\/1275898134942691330\">Steve Troughton-Smith<\/a>:<\/p>\n<blockquote cite=\"https:\/\/twitter.com\/stroughtonsmith\/status\/1275898134942691330\">\n<p>Incidentally, the new stripped framework cache on macOS 11 is <em>horrendous<\/em> for disassembly. If you&rsquo;re trying to track down why there&rsquo;s a bug in your app, or how a system implementation works, you are screwed. This is going to hurt developers more than the ARM transition<\/p>\n<\/blockquote>\n\n<p><a href=\"https:\/\/lapcatsoftware.com\/articles\/bigsur.html\">Jeff Johnson<\/a> (<a href=\"https:\/\/twitter.com\/lapcatsoftware\/status\/1275954993103847424\">tweet<\/a>, also: <a href=\"https:\/\/worthdoingbadly.com\/dscextract\/\">zhuowei<\/a>):<\/p>\n<blockquote cite=\"https:\/\/lapcatsoftware.com\/articles\/bigsur.html\"><p>If the libraries are no longer present on the filesystem, that makes it awfully hard to disassemble them! Fortunately, there are ways to extract the system libraries from the cache. One way is provided by Apple itself: the <code>dyld_shared_cache_util<\/code> command-line tool. Unfortunately, this tool does not come installed with macOS Big Sur. However, the tool is open source, so we can build it ourselves.<\/p><\/blockquote>\n\n<p><a href=\"https:\/\/lapcatsoftware.com\/articles\/bigsur2.html\">Jeff Johnson<\/a> (<a href=\"https:\/\/twitter.com\/lapcatsoftware\/status\/1276180661582299138\">tweet<\/a>):<\/p>\n<blockquote cite=\"https:\/\/lapcatsoftware.com\/articles\/bigsur2.html\"><p>Let&rsquo;s take a look at an example from my favorite framework, AppKit.<\/p>\n<p>[&#8230;]<\/p>\n<p>It seems that prior to Big Sur, Objective-C references in a Mach-O file are offsets from the beginning on the file, whereas on Big Sur, Objective-C references in a Mach-O file are offsets from the beginning of the dyld shared cache. Roughly speaking.<\/p><\/blockquote>\n\n<p>You can also point <a href=\"https:\/\/www.hopperapp.com\">Hopper<\/a> at the shared cache in the folder <tt>\/System\/Library\/dyld\/<\/tt>, and it will let you choose which library to load. But, as with <code>dyld_shared_cache_util<\/code>, what you end up with is difficult to work with because the tools don&rsquo;t know how to find the Objective-C selector information.<\/p>\n\n<p>Big Sur also adds another optimization that gets in the way of reverse engineering. <a href=\"https:\/\/twitter.com\/LeoNatan\/status\/1275439297533886464\">Leo Natan<\/a>:<\/p>\n<blockquote cite=\"https:\/\/twitter.com\/LeoNatan\/status\/1275439297533886464\">\n<p>A lot of Apple&rsquo;s private APIs are now peppered with direct and can no longer be swizzled.<\/p>\n<\/blockquote>\n\n<p>This makes it harder to debug and work around bugs. Unlike with the shared cache, this can&rsquo;t be worked around with better tools. The information (and indirection) have been removed from the library entirely.<\/p>\n\n<p>Previously:<\/p>\n<ul>\n<li><a href=\"https:\/\/mjtsai.com\/blog\/2020\/01\/20\/rejected-for-working-around-macos-bugs\/\">Rejected for Working Around macOS Bugs<\/a><\/li>\n<li><a href=\"https:\/\/mjtsai.com\/blog\/2019\/11\/19\/direct-objective-c-properties\/\">Direct Objective-C Properties<\/a><\/li>\n<\/ul>\n\n<p id=\"reverse-engineering-macos-11-0-update-2020-07-06\">Update (2020-07-06): <a href=\"https:\/\/twitter.com\/antons\/status\/1279344046104870912\">Anton Sotkov<\/a>:<\/p>\n<blockquote cite=\"https:\/\/twitter.com\/antons\/status\/1279344046104870912\">\n<p><a href=\"https:\/\/github.com\/antons\/dyld-shared-cache-big-sur\">Modifications<\/a> to Apple&rsquo;s dyld project to fix Objective-C information when extracting dyld_shared_cache from macOS Big Sur to help Hopper generate readable pseudocode.<\/p>\n<\/blockquote>\n\n<p id=\"reverse-engineering-macos-11-0-update-2020-07-27\">Update (2020-07-27): <a href=\"https:\/\/lapcatsoftware.com\/articles\/bigsur3.html\">Jeff Johnson<\/a>:<\/p>\n<blockquote cite=\"https:\/\/lapcatsoftware.com\/articles\/bigsur3.html\">\n<p>Static disassembly tools such as otool and llvm-objdump have not been updated to handle the dyld shared cache on Big Sur. However, one tool that does handle it is lldb, the debugger.<\/p>\n<p>[&#8230;]<\/p>\n<p>I hope that my little hack helps you to disassemble system libraries on Big Sur. It&rsquo;s a bid tedious, but it mostly works, and you only have to do it once for each library you&rsquo;re interested in.<\/p>\n<\/blockquote>\n\n<p>See also: <a href=\"https:\/\/mjtsai.com\/blog\/2020\/07\/27\/hopper-for-apple-silicon-and-big-sur\/\">Hopper for Apple Silicon and Big Sur<\/a><\/p>\n\n<p id=\"reverse-engineering-macos-11-0-update-2020-11-25\">Update (2020-11-25): <a href=\"https:\/\/twitter.com\/nevyn\/status\/1331217137889931269\">nevyn Bengtsson<\/a>:<\/p>\n<blockquote cite=\"https:\/\/twitter.com\/nevyn\/status\/1331217137889931269\">\n<p>Remember when I complained months ago how bad it would be for <em>everyone<\/em> if Big Sur really did ship without system library binaries? Less open, harder to develop, etc etc. Well, now even the latest version of CMake doesn&rsquo;t understand how to make Big Sur apps, it seems.<\/p>\n<p>Here&rsquo;s the solution. Replace <code>-framework AVFoundation<\/code> with <code>FIND_LIBRARIES<\/code>. Now it finds the <tt>.tbd<\/tt> and links with that INSTEAD.<\/p>\n<p>But I find it absurd to break decades-old foundations in every build tool that isn&rsquo;t Xcode &gt;12.2.<\/p>\n<\/blockquote>\n\n<p id=\"reverse-engineering-macos-11-0-update-2021-09-08\">Update (2021-09-08): <a href=\"https:\/\/twitter.com\/stroughtonsmith\/status\/1432723871078170627\">Steve Troughton-Smith<\/a>:<\/p>\n<blockquote cite=\"https:\/\/twitter.com\/stroughtonsmith\/status\/1432723871078170627\">\n<p>Apple&rsquo;s changes to how its system frameworks are packaged in macOS 11+, rendering them stripped &amp; extremely difficult to reverse-engineer, is proving disastrous for finding &amp; fixing framework-level bugs &#x1F62A;<\/p>\n<\/blockquote>\n\n<p id=\"reverse-engineering-macos-11-0-update-2024-02-01\">Update (2024-02-01): <a href=\"https:\/\/wadetregaskis.com\/reminder-macos-system-frameworks-binaries-are-hidden-since-big-sur\/\">Wade Tregaskis<\/a>:<\/p>\n<blockquote cite=\"https:\/\/wadetregaskis.com\/reminder-macos-system-frameworks-binaries-are-hidden-since-big-sur\/\"><p>The good news for Hopper is that it has since been updated to work around this &#x2013; you can access the Apple framework binaries through <em>File<\/em> &gt; <em>Read File from DYLD Cache&#8230;<\/em>  There&rsquo;s also tools like <a href=\"https:\/\/github.com\/keith\/dyld-shared-cache-extractor\">dyld-shared-cache-extractor<\/a> which can resurrect the binaries from the cache.<\/p><p>Note also that in Sonoma, at least, the cache lives at <tt>\/System\/Volumes\/Preboot\/Cryptexes\/OS\/System\/Library\/dyld\/<\/tt> (in previous macOS releases it was apparently in <tt>\/System\/Library\/dyld\/<\/tt>).<\/p><\/blockquote>","protected":false},"excerpt":{"rendered":"<p>Apple: New in macOS Big Sur 11 beta, the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"apple_news_api_created_at":"2020-06-26T20:16:34Z","apple_news_api_id":"d9f92fcc-81b2-45fb-8492-5d5679053c1d","apple_news_api_modified_at":"2024-02-01T15:02:03Z","apple_news_api_revision":"AAAAAAAAAAAAAAAAAAAABQ==","apple_news_api_share_url":"https:\/\/apple.news\/A2fkvzIGyRfuEkl1WeQU8HQ","apple_news_coverimage":0,"apple_news_coverimage_caption":"","apple_news_is_hidden":false,"apple_news_is_paid":false,"apple_news_is_preview":false,"apple_news_is_sponsored":false,"apple_news_maturity_rating":"","apple_news_metadata":"\"\"","apple_news_pullquote":"","apple_news_pullquote_position":"","apple_news_slug":"","apple_news_sections":"\"\"","apple_news_suppress_video_url":false,"apple_news_use_image_component":false,"footnotes":""},"categories":[4],"tags":[770,56,205,763,317,30,2792,1891,966,54,760,138,1013,71],"class_list":["post-29366","post","type-post","status-publish","format-standard","hentry","category-programming-category","tag-assembly-language","tag-debugging","tag-dyld","tag-hopper","tag-lldb","tag-mac","tag-mach-o","tag-macos-11-0","tag-message-passing","tag-objective-c","tag-objective-c-runtime","tag-optimization","tag-private-api","tag-programming"],"apple_news_notices":[],"_links":{"self":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/29366","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/comments?post=29366"}],"version-history":[{"count":6,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/29366\/revisions"}],"predecessor-version":[{"id":41955,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/29366\/revisions\/41955"}],"wp:attachment":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/media?parent=29366"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/categories?post=29366"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/tags?post=29366"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}