{"id":42256,"date":"2024-02-27T14:35:48","date_gmt":"2024-02-27T19:35:48","guid":{"rendered":"https:\/\/mjtsai.com\/blog\/?p=42256"},"modified":"2024-03-11T15:28:21","modified_gmt":"2024-03-11T19:28:21","slug":"slow-swift-macro-compilation","status":"publish","type":"post","link":"https:\/\/mjtsai.com\/blog\/2024\/02\/27\/slow-swift-macro-compilation\/","title":{"rendered":"Slow Swift Macro Compilation"},"content":{"rendered":"<p><a href=\"https:\/\/twitter.com\/vatsal_manot\/status\/1750238642763915738\">Vatsal Manot<\/a>:<\/p>\n<blockquote cite=\"https:\/\/twitter.com\/vatsal_manot\/status\/1750238642763915738\">\n<p>Folks, if you&rsquo;re also frustrated with the state of Swift Macros (w.r.t. compile times), please engage on the forums thread here.<\/p>\n<p>I can&rsquo;t think of any other language where such a fundamental feature is dependent on a heavy source package that needs to be recompiled every clean build.<\/p>\n<\/blockquote>\n\n<p><a href=\"https:\/\/forums.swift.org\/t\/compilation-extremely-slow-since-macros-adoption\/67921\/24\">John Bushnell<\/a>:<\/p>\n<blockquote cite=\"https:\/\/forums.swift.org\/t\/compilation-extremely-slow-since-macros-adoption\/67921\/24\"><p>Prior to adding macros one xcframework takes 257.7 seconds (4.3 minutes) to build. After adding macros it takes 2004.5 seconds (33.4 minutes). This is a 7.76x increase. This is with one macro implemented in one location in code, so it&rsquo;s not related to how many times the macro is used.<\/p><p>This speed decrease makes the macros support currently unusable for us.<\/p>\n<p>[&#8230;]<\/p><p>There&rsquo;s also a secondary issue, which because of the speed issue I don&rsquo;t really need to take it further at this point, but it&rsquo;s that tests do not support these macros for some reason when run from the command line.<\/p><\/blockquote>\n\n<p><a href=\"https:\/\/forums.swift.org\/t\/compilation-extremely-slow-since-macros-adoption\/67921\/32\">Wade Tregaskis<\/a>:<\/p>\n<blockquote cite=\"https:\/\/forums.swift.org\/t\/compilation-extremely-slow-since-macros-adoption\/67921\/32\"><p>Same story as everyone else, in a nutshell - I saw some nice-looking package (<a href=\"https:\/\/github.com\/CaptureContext\/swift-foundation-extensions\">swift-foundation-extensions<\/a>, in this instance), added it to my project, and then saw my builds suddenly taking <em>way<\/em> longer.  Then noticed the dreaded &ldquo;SwiftSyntax&rdquo; named in the build logs.<\/p><p>What was a clean build time of about fifteen seconds became several minutes.<\/p><p>(&ldquo;how often do you do clean builds, though, really?&rdquo; - ugh, a <em>lot<\/em>.  I&rsquo;m using Xcode and it&rsquo;s got some bugs regarding detecting changed sources, not to mention its infamous bugginess regarding stale compiler errors)<\/p><\/blockquote>\n\n<p><a href=\"https:\/\/forums.swift.org\/t\/compilation-extremely-slow-since-macros-adoption\/67921\/54\">John McCall<\/a>:<\/p>\n<blockquote cite=\"https:\/\/forums.swift.org\/t\/compilation-extremely-slow-since-macros-adoption\/67921\/54\">\n<p>The LSG is aware that there are build-system issues causing a lot of pain for macro adopters and their downstreams. From an abstract language perspective, I don&rsquo;t think any of those problems are particularly challenging to the point of requiring an overall design change to macros; the project just needs to put in the work to fix them. I can&rsquo;t tell you when that will happen, though, which I know is not a satisfying answer.<\/p>\n<\/blockquote>\n\n<p><a href=\"https:\/\/forums.swift.org\/t\/compilation-extremely-slow-since-macros-adoption\/67921\/68\">Vatsal Manot<\/a>:<\/p>\n<blockquote cite=\"https:\/\/forums.swift.org\/t\/compilation-extremely-slow-since-macros-adoption\/67921\/68\"><p>Be aware of the fact that you&rsquo;re adding ~38,000 LoC of Swift source code to your project. This cost is forwarded to anyone consuming your Swift packages.<\/p><p>Be aware of the fact that <a href=\"https:\/\/forums.swift.org\/t\/why-is-fetching-dependencies-with-swiftpm-so-slow\/67191\/10\">SPM currently fetches the entire git history of the SwiftSyntax repo<\/a>. This is a separate but, in my view, not unrelated issue as it directly factors into the cost of adopting macros.<\/p><p>Just adding SwiftSyntax can potentially add up to <a href=\"https:\/\/forums.swift.org\/t\/compilation-extremely-slow-since-macros-adoption\/67921\/34\"><strong>12 minutes<\/strong><\/a> to your build time on Xcode Cloud as noted by <a href=\"https:\/\/forums.swift.org\/u\/ignacio_soto\">@Ignacio_Soto<\/a>.<\/p><p>It&rsquo;s peculiar enough that a core language feature is tied to a still-maturing package manager riddled with performance\/build-systems issues, but what I find extremely disappointing is that these issues aren&rsquo;t mentioned <strong>anywhere<\/strong> in any official documentation\/release notes.<\/p><\/blockquote>\n\n<p>Previously:<\/p>\n<ul>\n<li><a href=\"https:\/\/mjtsai.com\/blog\/2023\/07\/17\/macros-in-swift-5-9\/\">Macros in Swift 5.9<\/a><\/li>\n<\/ul>\n\n<p id=\"slow-swift-macro-compilation-update-2024-02-28\">Update (2024-02-28): <a href=\"https:\/\/mastodon.social\/@pilky\/111697415616780882\">Martin Pilkington<\/a>:<\/p>\n<blockquote cite=\"https:\/\/mastodon.social\/@pilky\/111697415616780882\">\n<p>A clean build of my CoppiceCore framework &amp; tests now takes 2m36s, only about 10s of which is my code<\/p>\n<p>Apple really needs to switch to Macros linking against a pre-build version included in Swift distributions if they want Macros to be a feature many people use<\/p>\n<\/blockquote>\n\n<p id=\"slow-swift-macro-compilation-update-2024-03-01\">Update (2024-03-01): <a href=\"https:\/\/mastodon.social\/@alloca@techhub.social\/112014806839678094\">alloca<\/a>:<\/p>\n<blockquote cite=\"https:\/\/mastodon.social\/@alloca@techhub.social\/112014806839678094\"><p>I feel like I&rsquo;ve seen clean build times increase by maybe 30-60 seconds on my M1 Pro when adding macros. Am I missing something? How are people getting several minute increases?<\/p><\/blockquote>\n\n<p><a href=\"https:\/\/forums.swift.org\/t\/macro-adoption-concerns-around-swiftsyntax\/66588\">Stephen Celis<\/a> (via <a href=\"https:\/\/mastodon.social\/@helge\/111059787261598163\">Helge He&szlig;<\/a>):<\/p>\n<blockquote cite=\"https:\/\/forums.swift.org\/t\/macro-adoption-concerns-around-swiftsyntax\/66588\"><p>Introducing SwiftSyntax to a project immediately incurs an additional 20 second debug build cost to a project, which may not seem like much, but that&rsquo;s an extra minute for every 3 cleans. Things get much slower when building for release with whole module optimization: over 4 minutes just for SwiftSyntax. <a href=\"https:\/\/github.com\/apple\/swift-syntax\/issues\/1194\">This issue<\/a> from January highlights the problem, but there isn&rsquo;t a lot of discussion around how it might be addressed.<\/p>\n<p>[&#8230;]<\/p>\n<p>Beyond build times, SwiftSyntax is a complex project to depend on, and it&rsquo;s unclear how to version a project that depends on it. I started <a href=\"https:\/\/forums.swift.org\/t\/versioning-libraries-that-use-macros-and-swiftsyntax\/66514\">this discussion<\/a> a few days ago and was hoping for guidance there, but I&rsquo;ll restate the problem here. Because SwiftSyntax is a moving target and versioned alongside Swift releases, how can a library adopt macros and be compatible with multiple Swift versions at the same time?<\/p><\/blockquote>\n\n<p id=\"slow-swift-macro-compilation-update-2024-03-11\">Update (2024-03-11): <a href=\"https:\/\/twitter.com\/vatsal_manot\/status\/1766498480028279285\">Vatsal Manot<\/a>:<\/p>\n<blockquote cite=\"https:\/\/twitter.com\/vatsal_manot\/status\/1766498480028279285\"><p>Friendly reminder that there&rsquo;s still no ETA on when the massive compilation time issue for Swift Macros will be fixed.<\/p><p>I&rsquo;d personally recommend holding off on adoption until there&rsquo;s an official update from the team on this.<\/p><\/blockquote>\n\n<p><a href=\"https:\/\/twitter.com\/_saagarjha\/status\/1766469012601614463\">Saagar Jha<\/a>:<\/p>\n<blockquote cite=\"https:\/\/twitter.com\/_saagarjha\/status\/1766469012601614463\">\n<p>As I watch complaints mount about swift-syntax compile times, I wonder if Apple remembers the time when them ignoring the concerns of impatient developers led to the infection of hundreds of millions of iOS devices in one of the most successful supply-chain backdoor attacks ever<\/p>\n<p>Nobody&rsquo;s going to do the whole Reflections on Trusting Trust thing but the longer it takes for your project to build the easier it gets for me to offer you my precompiled binaries instead. And interestingly enough these binaries happen to have direct control of code generation&#8230;<\/p>\n<\/blockquote>","protected":false},"excerpt":{"rendered":"<p>Vatsal Manot: Folks, if you&rsquo;re also frustrated with the state of Swift Macros (w.r.t. compile times), please engage on the forums thread here. I can&rsquo;t think of any other language where such a fundamental feature is dependent on a heavy source package that needs to be recompiled every clean build. John Bushnell: Prior to adding [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"apple_news_api_created_at":"2024-02-27T19:35:50Z","apple_news_api_id":"1a51dd64-d5cc-40f4-8854-e26b6de57a0a","apple_news_api_modified_at":"2024-03-11T19:28:24Z","apple_news_api_revision":"AAAAAAAAAAAAAAAAAAAABA==","apple_news_api_share_url":"https:\/\/apple.news\/AGlHdZNXMQPSIVOJrbeV6Cg","apple_news_coverimage":0,"apple_news_coverimage_caption":"","apple_news_is_hidden":false,"apple_news_is_paid":false,"apple_news_is_preview":false,"apple_news_is_sponsored":false,"apple_news_maturity_rating":"","apple_news_metadata":"\"\"","apple_news_pullquote":"","apple_news_pullquote_position":"","apple_news_slug":"","apple_news_sections":"\"\"","apple_news_suppress_video_url":false,"apple_news_use_image_component":false,"footnotes":""},"categories":[4],"tags":[255,941,71,901,268,226],"class_list":["post-42256","post","type-post","status-publish","format-standard","hentry","category-programming-category","tag-compiler","tag-macros","tag-programming","tag-swift-programming-language","tag-testing","tag-xcode"],"apple_news_notices":[],"_links":{"self":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/42256","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/comments?post=42256"}],"version-history":[{"count":6,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/42256\/revisions"}],"predecessor-version":[{"id":42456,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/42256\/revisions\/42456"}],"wp:attachment":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/media?parent=42256"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/categories?post=42256"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/tags?post=42256"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}