Archive for June 24, 2024

Monday, June 24, 2024

Apple Found in Breach of DMA

Lisa O’Carroll (via Hacker News, New York Times, Slashdot):

Apple has been found to be in breach of sweeping new EU laws designed to allow smaller companies to compete and allow consumers to find cheaper and alternative apps in the tech business’s app store.

The European Commission, which also acts as the EU antitrust and technology regulator, said it had sent its preliminary findings to Apple after an investigation launched in March.


The company has 12 months to comply before it face fines of up to 10% of its global revenues but the EU hopes ongoing dialogue will lead to compliance rather than sanctions.

Margrethe Vestager (tweet):

Because the ball is now in the gatekeepers’ court. They have to convince us that the measures they take will achieve full compliance with the DMA. And where this is not the case, we will intervene. Within a month of the compliance deadline, we opened no less than five non-compliance cases. Today, we are opening a sixth one: we will look into Apple’s new business model: the commercial terms Apple imposes on app developers who want to reach end users on the iOS platform. The criteria these app developers have to meet to even be allowed to operate as alternative marketplaces or make apps available via sideloading. And the complex user journey for those users who want to download and install alternative marketplaces and sideloaded apps.

We are concerned that Apple designed its new business model to discourage app developers and end users from taking advantage of the opportunities afforded to them by the DMA. The letter of the DMA is clear: gatekeepers have to allow for alternative app stores to establish themselves on their platforms; and for consumers to be fully informed about the offers available to them. So that they can freely choose where they want to source their apps, and at what conditions.

And there is more. Today, and after less than three months from opening, we adopt our first Preliminary Findings in a case of non-compliance. And it is again about Apple. About the many ways in which their new terms fall short of the DMA requirements regarding steering of users to options outside the Apple App Store. As they stand, we think that these new terms do not allow app developers to communicate freely with their end users, and to conclude contracts with them.

William Gallagher and Mike Wuerthele:

In a statement to AppleInsider, Apple said that it denies failing to comply with the DMA.

Throughout the past several months, Apple has made a number of changes to comply with the DMA in response to feedback from developers and the European Commission. We are confident our plan complies with the law, and estimate more than 99% of developers would pay the same or less in fees to Apple under the new business terms we created.

All developers doing business in the EU on the App Store have the opportunity to utilize the capabilities that we have introduced, including the ability to direct app users to the web to complete purchases at a very competitive rate. As we have done routinely, we will continue to listen and engage with the European Commission.

John Voorhees:

In particular, the EC’s preliminary findings take issue with Apple’s response to the DMA’s anti-steering provisions[…]

Benjamin Mayo:

The Commission says Apple may charge a fee for facilitating “the initial acquisition of a new customer” via the App Store, but it essentially cannot charge for each ongoing transaction.

Tim Hardwick:

The Commission also said it was opening a new non-compliance procedure against Apple over concerns that its new contractual requirements for third-party app developers and app marketplaces, including its €0.50 Core Technology Fee, “fall short of ensuring effective compliance with Apple’s obligations under the DMA.”

Dan Moren:

Should the EC find Apple to not be in compliance in these areas, it would require a substantial reworking of much of Apple’s EU terms. As with the previous investigations, it will likely take some time for a final ruling to be issued, though we may get a preliminary ruling such as the one above in a matter of months.

This uncertainty is not good for anyone trying to build a business using App Marketplaces or Web Distribution.

Warner Crocker:

This will bounce back and forth over the next nine months and will probably become even more contentious given quotes like this from Thierry Breton, the EU internal market commissioner: “Apple’s new slogan should be ‘act different.’”


Update (2024-06-25): Jesper:

In the DMA, the ground rule is for sideloading apps to be allowed, and to only very minimally be reigned in under very specific conditions. Apple chose to take these conditions and lawyer them into “always, unless you pay us sums of money that are plainly prohibitive for most actors”. Apple knew the rules and understood the intent and chose to evade them, in order to retain additional income.

In the App Store Guidelines, as written and period appropriate, the ground rule was for in-app purchases to be allowed only through the App Store’s native in-app purchase system, at the penalty of removal from the App Store. Epic chose to take those conditions, break them and lawyer up. Epic knew the rules and understood the intent and chose to evade them, in order to retain additional income.

It is completely fair to look at what Epic did and say “that was kind of a dick move”. (I personally think it was kind of a dick move, even as I agree with some downwind consequences.) But any argument that what Epic did was wrong and what Apple did was right hinges on distinctions that do not make sense to me.


Apple has a significantly easier time silently assenting to the qualms of dictatorships than to simply stop reaching into the pockets of customers, many of which have funneled tens to hundreds of thousands of dollars for the most consistently successful high margin product in the history of mobile telephony, or developers who have largely made those devices attractive in the first place.

John Gruber:

This sounds like they’re going to insist that Apple make installing sideloaded apps and alternative stores a no-hassle experience. What critics see is Apple putting up obstacles to installing marketplaces or sideloaded apps just to be a dick about it and discouraging their use to keep users in the App Store. What I see are reasonable warnings for potentially dangerous software. We’ll see how that goes.


For sideloading, Apple requires that developers “Be a member in good standing of the Apple Developer Program for two continuous years or more, and have an app that had more than one million first annual installs on iOS and/or iPadOS in the EU in the prior calendar year.” Apple’s requirements are an attempt to prevent fly-by-night scammers from opening marketplaces or offering nefarious apps for sideloading. But the EC sees that as a catch-22, where the only way to become a marketplace or offer sideloading is to already be a longstanding developer in Apple’s own App Store.


I complain as much as anyone about the aspects of the DMA that are vague (or downright inscrutable), but this aspect seems clear-cut. It’s a bit baffling why Apple seemingly sees notarization as an opportunity for content/purpose review, like with last week’s brouhaha over the UTM SE PC emulator.

Riley Testut:

We tried to warn Apple that rejecting UTM was illegal 😬

When we first met with the EC a few months ago, we were asked repeatedly if we trusted Apple to be in charge of Notarization. We emphatically said yes.

However, it’s clear to us now that Apple is indeed using Notarization to not only delay our apps, but also to determine on a case-by-case basis how to undermine each release — such as by changing the App Store rules to allow them.

For these reasons, we are no longer telling the EC we trust Apple to be in charge of Notarization. 🤷‍♂️

Paul Haddad:

I’m confused as to why the answer was an emphatic yes to begin with. At best the answer should’ve been Apple hasn’t weaponized notarization on Mac, yet…

Steve Troughton-Smith:

Apple is convinced

1) that they’re the good guys
2) that European regulators are dumb

It’s patently obvious that neither of these are the case. Until that actually sinks in, this is going to continue to be a rollercoaster.

Michael Love:

On Apple and the EU: if you take all of our EU revenue on iOS and move it to Android instead, it would be enough to make our Android revenue roughly even with our iOS revenue. In absolute terms it’s like 20% of iOS, but if you take 20% of iOS and move it to Android, Android ends up bigger than iOS, at least for us.

So from the standpoint of developer platform priorities, this is an extremely stupid game for Apple to play, and they stand to lose far, far more from it than from the DMA.

Update (2024-06-26): Nick Heer:

If you are somebody who believes it is only fair to take someone at their word and assume good faith, I am right there with you. Even though Apple has a long history of capricious App Review processes, it was fair to consider its approach to the E.U. a begrudging but earnest attempt at compliance.


That is, however, a rather difficult position to maintain, given the growing evidence Apple seems determined to evade both the letter and spirit of this legislation.

Update (2024-07-02): Steve Troughton-Smith:

I think how the EU intends to enforce & litigate the DMA has become much clearer over the past couple weeks[…]

SwiftData vs. Realm Performance Comparison

Jacob Bartlett:

The Realm DB engine was written from the ground-up in C++ to minimise this overhead. […] Therefore, it’s not unreasonable to describe SwiftData as a wrapper over a wrapper over a wrapper.


These show that the SwiftData objects took around 10x longer to instantiate.


Realm topped out at writing 2,000,000 simple User objects before hitting an out-of-memory exception and crashing. SwiftData was only able to manage a paltry 1,000,000 objects. Realm was about 3-6x faster beyond write volumes exceededing 1,000 objects.


For our dead-simple User objects (a few fields and no relationships), we queried for all users with the first name “Jane”. Realm was much faster, its zero-copy architecture shining when reading data directly into memory. For simple SwiftData objects, read performance started off okay and degraded sharply with over 100k objects in the database.

With our more complex Student model, we searched for all Physics students who got the top grade. We observed the opposite effect: SwiftData was usually more than 10x faster than Realm.

Interestingly, SwiftData sometimes has the edge with smaller datasets, both in terms of RAM use and speed.


Update (2024-06-26): Wade Tregaskis:

I think we (Shark engineers) tried to be open-minded and kind. We were sceptical, but you never know until you actually look. We could see some potential for a more general query capability, for example. But of course the first and most obvious hurdle was: how well does Core Data handle sizeable numbers of records? Oh yes, was the response, it’s great even with tens of thousands of records.


We asked how it did with tens of millions of records, and that was pretty much the end of the conversation.


I guess by modern standards SQLite is considered efficient and fast, but – hah – back in my day SQLite was what you used when you didn’t have time to write your own, efficient and fast persistent data management system.

Helge Heß:

The Realm vs SwiftData thing encouraged me to try the same project w/ Lighter. It isn’t exactly the same as Lighter is no ORM, but it will give some hints on what a little lower level can yield. For now, the 10k plain items test:

💽 User instantiation: 0.0676
💽 Create users: 1.9151

💽 User instantiation: 0.0229
💽 Create users: 0.1220

💽 User instantiation: 0.0049
💽 Create users: 0.0820


Something disappointing in SwiftData is that it doesn’t make use of the static nature of the macro(s). The macro can’t see the full schema like Lighter does, but it could still statically generate a ton, e.g. a static snapshot struct for the backing data. Or predefined indices for quickly binding the snapshot to the SQLite API (or really any).

Instead we get custom backends.

Helge Heß:

So I’ve essentially ported the whole perf test over to Lighter, which was interesting because it also demonstrates some key differences in the approaches. E.g. to update the math grades of the bad students, the sample essentially loads all students and their grades into memory, then updates the grades one-by-one and saves them back to the store.

In plain SQL that would be just a single line UPDATE statement, no loading at all.

The SwiftData implementation in the test also seems to be not quite optimal. E.g. to update the items, already fetched items get inserted into a new ModelContext and then saved (a lot of new MCs are created, which seems completely counter the idea, though sometimes necessary to keep SwiftData RAM at bounds). Presumably just saving the context used to fetch the items would be quite a bit more efficient.

Update (2024-07-01): Aleksandar Vacić (Mastodon):

Thus by using somewhat reasonable but still sizable chunks (100k records) of the original data set and employing Core Data best practices, we lowered peak memory usage 10× and shortened total time spent about 2.5× which is no small feat.

Always Allow Safari Bookmarklets

Jeff Johnson:

You may already be aware that for a number of years, Safari has asked your permission every time you click on a link, such as an RSS feed, that opens in an app other than Safari[…]


The permission prompt now has an option to “Always Allow”! This option is new in Safari 17.

This is an improvement, but even with Always Allow it only remembers per-domain. So I’m still prompted a lot when using MarsEdit and EagleFiler bookmarklets. And it messes up my muscle memory because I had been in the habit of always pressing Enter after invoking a bookmarklet to Allow it—Can you see from the slightly bold text in this iOS-style alert that Allow is the default button?—but with Always Allow it’s unpredictable. I have to either pause to see whether I need to press Enter or I end up with an extra blank line in my blog post draft.

You might wonder where this new preference is stored on disk. As far as I can tell, there’s no corresponding user interface in Safari Settings, certainly not in the Websites pane. What if you want to undo your selection? What if you select Always Allow by accident?


The good news is that with a little reverse engineering, I found a way to undo the preference. It’s stored on disk in the file ~/Library/Safari/PerSitePreferences.db, which is an SQLite database.

Jeff Johnson (Mastodon):

To run bookmarklets in Safari on macOS, you need to enable “Show features for web developers” in Safari Advanced Settings and “Allow JavaScript from Smart Search field” in Safari Developer Settings.

I think this is only necessary for testing them. I have this unchecked, and my previously created bookmarklets still work.

The permission is per-website, which means that every time you use the EagleFiler bookmarklet on a different website, Safari requests your permission again!

But he has a workaround:

This JavaScript first calls, which creates a new about:blank tab. It then creates an HTML anchor element—in other words, a hyperlink—adds the link to the about:blank document, and clicks the link automatically.


This time the value of the domain is empty (''), because about:blank has no domain. The about:blank trick allows you to use the same bookmarklet on every website without any additional permission prompts!

This kind of exposes the permission prompt as security theater, but if it’s not protecting us anyway we may as well get rid of the annoyance.


iCloud Drive, Dropbox, and Proton Drive

Ryan Christoffel (AppleInsider, MacRumors):

The problem is, Apple “intelligently” decides which files can remain stored in local cache, and will make decisions to remove certain downloads without telling you. So when you need to access a given file—say, on an airplane with no connection—you might find that the file has been sent back to the cloud and is no longer available.

iPadOS 18 changes that.

Not only on the iPad but also the iPhone in iOS 18, you can long-press on a file or folder and find a new ‘Keep Downloaded’ button in the menu.

Francisco Tolmasky:

Something lost in this “Dropbox is a feature, not a product” story is that today, more than 10 years later, iCloud Drive (Apple’s implementation of this “feature”) still sucks. And Dropbox is arguably only falling behind because macOS has made it increasingly difficult to make a 3rd party syncing solution that “just works.” So maybe the real lesson is that Apple, like Game of Thrones’ Littlefinger, “would see the [OS] burn, as long as they can be king of the ashes”.

The two main problems I’ve had with iCloud Drive are files not uploading promptly and needless eviction of recently downloaded files that I want to keep. The former seems to have gotten better over the last year, and it sounds like iOS 18 and Sequoia will address the latter.

Howard Oakley:

Unexpected behaviour is seen when the user turns off the setting to put Desktop & Documents Folders into iCloud Drive. Instead of moving the folders back from the iCloud Drive location to their original location, macOS creates fresh and empty folders in the regular Home folder. Although the contents of the previous Desktop and Documents folders are retained in iCloud Drive, when seen from their Mac, the user may believe that those entire contents have been deleted, despite an alert that tries to explain what will happen. At least this time the user is offered a way back to reconsider their action, although it’s unclear what other option they might have.

When turned off, those folders are removed with all their contents, which remain in iCloud Drive, but are absent from the empty local Documents and Desktop folders.

The saving grace to this counter-intuitive behaviour is that, despite their apparent movement, the files themselves have remained within the same volume throughout the process of ‘moving’ to iCloud Drive, and ‘vanishing’ on their return journey. As they retain the same inode numbers at each stage of these processes, when they’re finally ‘moved’ manually back into ~/Documents and ~/Desktop, they have remained intact, complete with all their extended attributes and any saved versions. Thus their ‘movements’ preserve both data and metadata at all times.

Tim Hardwick:

Apple wants iPhone and iPad users to be able to format external drives connected to their device, without the need for a Mac, based on the latest find in the iOS 18 and iPadOS 18 developer betas (via MacStories).


Anyway, I gave Proton Drive a real go but from the get go, it was disappointing at best. It took three attempts to transfer 6GB files from Dropbox to Proton Drive. The upload and download speeds was terrible. At this point, I went seeking for answers on Reddit and what I discovered was anecdotes from many Proton users on Proton Drive is just not ready. I should have done my research first.


AI Companies Ignoring Robots.txt

Mark Sullivan:

The AI search startup Perplexity is in hot water in the wake of a Wired investigation revealing that the startup has been crawling content from websites that don’t want to be crawled.


“Perplexity is not ignoring the Robot Exclusions Protocol and then lying about it,” said Perplexity cofounder and CEO Aravind Srinivas in a phone interview Friday. “I think there is a basic misunderstanding of the way this works,” Srinivas said. “We don’t just rely on our own web crawlers, we rely on third-party web crawlers as well.”

Srinivas said the mysterious web crawler that Wired identified was not owned by Perplexity, but by a third-party provider of web crawling and indexing services. Srinivas would not say the name of the third-party provider, citing a Nondisclosure Agreement. Asked if Perplexity immediately called the third-parter crawler to tell them to stop crawling Wired content, Srinivas was non-committal. “It’s complicated,” he said.

Srinivas also noted that the Robot Exclusion Protocol, which was first proposed in 1994, is “not a legal framework.” He suggested that the emergence of AI requires a new kind of working relationship between content creators, or publishers, and sites like his.

Nick Heer (Mastodon, Hacker News):

Srinivas is creating a clear difference between laws and principles because the legal implications are so far undecided, but it sure looks unethical that its service ignores the requests of publishers — no matter whether that is through first- or third-party means.

Tim Marchman:

Earlier this week, WIRED published a story about the AI-powered search startup Perplexity, which Forbes has accused of plagiarism. In it, my colleague Dhruv Mehrotra and I reported that the company was surreptitiously scraping, using crawlers to visit and download parts of websites from which developers had tried to block it, in violation of its own publicly stated policy of honoring the Robots Exclusion Protocol.


After we published the story, I prompted three leading chatbots to tell me about the story. OpenAI’s ChatGPT and Anthropic’s Claude generated text offering hypotheses about the story’s subject but noted that they had no access to the article. The Perplexity chatbot produced a six-paragraph, 287-word text closely summarizing the conclusions of the story and the evidence used to reach them. (According to WIRED’s server logs, the same bot observed in our and Knight’s findings, which is almost certainly linked to Perplexity but is not in its publicly listed IP range, attempted to access the article the day it was published, but was met with a 404 response. The company doesn’t retain all its traffic logs, so this is not necessarily a complete picture of the bot’s activity, or that of other Perplexity agents.) The original story is linked at the top of the generated text, and a small gray circle links out to the original following each of the last five paragraphs. The last third of the fifth paragraph exactly reproduces a sentence from the original: “Instead, it invented a story about a young girl named Amelia who follows a trail of glowing mushrooms in a magical forest called Whisper Woods.”

This struck me and my colleagues as plagiarism.

Kali Hays (via John Voorhees):

OpenAI and Anthropic have said publicly they respect robots.txt and blocks to their web crawlers.

Yet, both companies are ignoring or circumventing such blocks, BI has learned.

Katie Paul:

TollBit said its analytics indicate “numerous” AI agents are bypassing the protocol, a standard tool used by publishers to indicate which parts of its site can be crawled.

“What this means in practical terms is that AI agents from multiple sources (not just one company) are opting to bypass the robots.txt protocol to retrieve content from sites,” TollBit wrote. “The more publisher logs we ingest, the more this pattern emerges.”


Update (2024-06-28): Elizabeth Lopatto:

“Someone else did it” is a fine argument for a five-year-old. And consider the response further. If Srinivas wanted to be ethical, he had some options here. Option one is to terminate the contract with the third-party scraper. Option two is to try to convince the scraper to honor robots.txt. Srinivas didn’t commit to either, and it seems to me, there’s a clear reason why. Even if Perplexity itself isn’t violating the code, it is reliant on someone else violating the code for its “answer engine” to work.

Update (2024-07-05): See also: Accidental Tech Podcast.

Pirate Ship

Adam Engst:

Pirate Ship is a shipping platform with an elegant interface that allows users to access discounted shipping rates from USPS and UPS with no subscription fee. I’ve used it a handful of times for mailing packages, and it has been brilliant.


And, oh, what a lovely interface!


Pirate Ship has negotiated corporate-level discounted shipping rates of up to 89% off retail and passes most of those savings on to customers. For shipping something heavy, Glenn has seen international shipping prices that run about $200 on UPS’s site, while Pirate Ship’s rate was about $60.

That slight arbitrage allows Pirate Ship to avoid the monthly subscriptions that make no sense for all but high-volume shippers— charges $19.99 per month plus postage, for example.


Pirate Ship’s support pages are also outstanding.