Archive for February 27, 2024

Tuesday, February 27, 2024

The “everything” NPM Package

Feross Aboukhadijeh (Hacker News):

Just when we thought we’d seen it all, an npm user named PatrickJS, aka gdi2290, threw us a curveball. He (along with a group of contributors) kicked off the year with a bang, launching a troll campaign that uploaded an npm package aptly named everything. This package, true to its name, depends on every other public npm package, creating millions of transitive dependencies.


The “everything” package, with its 5 sub-packages and thousands of dependencies, has essentially locked down the ability for authors to unpublish their packages. This situation is due to npm’s policy shift following the infamous “left-pad” incident in 2016, where a popular package left-pad was removed, grinding development to a halt across much of the developer world. In response, npm tightened its rules around unpublishing, specifically preventing the unpublishing of any package that is used by another package.

Ironically, this policy trapped PatrickJS in his own web. Upon realizing the impact of his prank, he attempted to remove the everything package but was unable to do so. He reached out to the npm support team for help, but the damage was done.


Reddit AI Training Data and IPO

Ben Lovejoy:

It’s being reported that a deal has been struck to allow an unnamed large AI company to use Reddit user content for training purposes …

The deal is said to be worth around $60M per year, and comes at a time when the company is seeking to maximize its value in the run-up to an IPO.

Emma Roth:

Google is getting AI training data from Reddit as part of a new partnership between the two companies. In an update on Thursday, Reddit announced it will start providing Google “more efficient ways to train models.”


The partnership will give Reddit access to Vertex AI as well, Google’s AI-powered service that’s supposed to help companies improve their search results. Reddit says the change doesn’t affect the company’s data API terms, which prevent developers or companies from accessing it for commercial purposes without approval.

Dare Obasanjo:

Called it. Only question is whether the deal is exclusive or if there’s an option to make it exclusive since $60M is relatively low for exclusivity.

Jonathan Vanian:

Social media company Reddit filed its IPO prospectus with the Securities and Exchange Commission on Thursday after a yearslong run-up. The company plans to trade on the New York Stock Exchange under the ticker symbol “RDDT.”


Reddit said it had $804 million in annual sales for 2023, up 20% from the $666.7 million it brought in the previous year, according to the filing. The social networking company’s core business is reliant on online advertising sales stemming from its website and mobile app.

Via Dare Obasanjo:

It was valued at $10B in its last funding round and Sam Altman owns an 8.7% stake.

A few surprises in Reddit’s financials are that it’s still unprofitable in its nineteenth year of existence and if it does list at $5B as rumored then it will be a down round relative to its series F funding round.

Emma Roth (via Matthew Cassinelli):

In an unusual twist, Reddit is also giving an unspecified number of its top users, including moderators and those with high karma scores, the chance to buy shares in its IPO. That’s a privilege usually reserved for professional investors who want to buy stock at a theoretically lower price before everyone else gets to purchase it on the public market.


Update (2024-03-11): Amy Or, Priya Anand, and Ryan Gould (via Hacker News):

Reddit Inc. disclosed further details of what is set to be one of the year’s biggest initial public offerings, with the company and some existing shareholders seeking to raise as much as $748 million.

Update (2024-03-20): Brandon Vigliarolo (Hacker News):

Ahead of its stock market debut, Reddit has said it’s going to make it easier for advertisers to craft normal user posts and run them as ads on the social network.

Called “free-form ads,” Reddit claimed in a statement this week that the “all-new, completely unique ad format” is designed to improve click-through rates and upvotes – by making adverts look as much like user posts as possible.

To be clear, we understand these free-form ads will be labeled as promoted posts.

Update (2024-03-22): Paul Graham (via Hacker News):

I met the Reddits before we even started Y Combinator. In fact they were one of the reasons we started it.


I don’t think the startup sense of the word “pivot” had been invented yet, but we wanted to fund Steve and Alexis, so if their idea was bad, they’d have to work on something else.


Reddit the site (and now app) is such a fundamentally useful thing that it’s almost unkillable.

Ianna Rosa Royle (via Hacker News):

While Reddit’s CEO Steve Huffman has been given a $193 million compensation package, the moderators on the platform are unpaid—and the disparity has not gone unnoticed. Since word got out about Huffman’s pay, users on Quora, Reddit, and X have been ripping into the package.

Laura Bratton:

The staggering figure was included in the company’s U.S. Securities and Exchange Commission (SEC) filing on Feb. 22 as it prepares to launch an initial public offering this week. Huffman defended his compensation in a video on his social media platform this week. Quartz calculates that it’s more than the pay packages for the CEOs of Pinterest, Snap, and Meta — combined.

Ines Ferré:

The stock closed at $50.44 on Thursday, up 48% from the IPO price and giving the company a market cap just north of $8 billion.

Slow Swift Macro Compilation

Vatsal Manot:

Folks, if you’re also frustrated with the state of Swift Macros (w.r.t. compile times), please engage on the forums thread here.

I can’t think of any other language where such a fundamental feature is dependent on a heavy source package that needs to be recompiled every clean build.

John Bushnell:

Prior to adding macros one xcframework takes 257.7 seconds (4.3 minutes) to build. After adding macros it takes 2004.5 seconds (33.4 minutes). This is a 7.76x increase. This is with one macro implemented in one location in code, so it’s not related to how many times the macro is used.

This speed decrease makes the macros support currently unusable for us.


There’s also a secondary issue, which because of the speed issue I don’t really need to take it further at this point, but it’s that tests do not support these macros for some reason when run from the command line.

Wade Tregaskis:

Same story as everyone else, in a nutshell - I saw some nice-looking package (swift-foundation-extensions, in this instance), added it to my project, and then saw my builds suddenly taking way longer. Then noticed the dreaded “SwiftSyntax” named in the build logs.

What was a clean build time of about fifteen seconds became several minutes.

(“how often do you do clean builds, though, really?” - ugh, a lot. I’m using Xcode and it’s got some bugs regarding detecting changed sources, not to mention its infamous bugginess regarding stale compiler errors)

John McCall:

The LSG is aware that there are build-system issues causing a lot of pain for macro adopters and their downstreams. From an abstract language perspective, I don’t think any of those problems are particularly challenging to the point of requiring an overall design change to macros; the project just needs to put in the work to fix them. I can’t tell you when that will happen, though, which I know is not a satisfying answer.

Vatsal Manot:

Be aware of the fact that you’re adding ~38,000 LoC of Swift source code to your project. This cost is forwarded to anyone consuming your Swift packages.

Be aware of the fact that SPM currently fetches the entire git history of the SwiftSyntax repo. This is a separate but, in my view, not unrelated issue as it directly factors into the cost of adopting macros.

Just adding SwiftSyntax can potentially add up to 12 minutes to your build time on Xcode Cloud as noted by @Ignacio_Soto.

It’s peculiar enough that a core language feature is tied to a still-maturing package manager riddled with performance/build-systems issues, but what I find extremely disappointing is that these issues aren’t mentioned anywhere in any official documentation/release notes.


Update (2024-02-28): Martin Pilkington:

A clean build of my CoppiceCore framework & tests now takes 2m36s, only about 10s of which is my code

Apple really needs to switch to Macros linking against a pre-build version included in Swift distributions if they want Macros to be a feature many people use

Update (2024-03-01): alloca:

I feel like I’ve seen clean build times increase by maybe 30-60 seconds on my M1 Pro when adding macros. Am I missing something? How are people getting several minute increases?

Stephen Celis (via Helge Heß):

Introducing SwiftSyntax to a project immediately incurs an additional 20 second debug build cost to a project, which may not seem like much, but that’s an extra minute for every 3 cleans. Things get much slower when building for release with whole module optimization: over 4 minutes just for SwiftSyntax. This issue from January highlights the problem, but there isn’t a lot of discussion around how it might be addressed.


Beyond build times, SwiftSyntax is a complex project to depend on, and it’s unclear how to version a project that depends on it. I started this discussion a few days ago and was hoping for guidance there, but I’ll restate the problem here. Because SwiftSyntax is a moving target and versioned alongside Swift releases, how can a library adopt macros and be compatible with multiple Swift versions at the same time?

Update (2024-03-11): Vatsal Manot:

Friendly reminder that there’s still no ETA on when the massive compilation time issue for Swift Macros will be fixed.

I’d personally recommend holding off on adoption until there’s an official update from the team on this.

Saagar Jha:

As I watch complaints mount about swift-syntax compile times, I wonder if Apple remembers the time when them ignoring the concerns of impatient developers led to the infection of hundreds of millions of iOS devices in one of the most successful supply-chain backdoor attacks ever

Nobody’s going to do the whole Reflections on Trusting Trust thing but the longer it takes for your project to build the easier it gets for me to offer you my precompiled binaries instead. And interestingly enough these binaries happen to have direct control of code generation…