Tuesday, January 21, 2020 [Tweets] [Favorites]

Apple Dropped Plans for End-to-End Encrypted iCloud Backups After FBI Objected

Benjamin Mayo:

In a 2018 interview, Cook’s comments to a German website heavily implied they are working on iCloud backups without a key (i.e. end-to-end encrypted).

Tim Cook (translation):

Our users have a key there, and we have one. We do this because some users lose or forget their key and then expect help from us to get their data back. It is difficult to estimate when we will change this practice. But I think that in the future it will be regulated like the devices. We will therefore no longer have a key for this in the future.

Joseph Menn (MacRumors, Hacker News):

Apple Inc dropped plans to let iPhone users fully encrypt backups of their devices in the company’s iCloud service after the FBI complained that the move would harm investigations, six sources familiar with the matter told Reuters.

The tech giant’s reversal, about two years ago, has not previously been reported. It shows how much Apple has been willing to help U.S. law enforcement and intelligence agencies, despite taking a harder line in high-profile legal disputes with the government and casting itself as a defender of its customers’ information.

Two years ago would be about 9 months before that interview with Cook was published.

As Apple explains, most iCloud data is not end-to-end encrypted. And, despite Apple’s marketing, iMessage has effectively never been end-to-end encrypted, either, unless all the parties in a conversation have turned off iCloud backup. I’ve been writing about this for years, but most reporting continues to ignore the backup loophole.

In October 2018, Alphabet Inc’s Google announced a similar system to Apple’s dropped plan for secure backups. The maker of Android software, which runs on about three-quarters of the world’s mobile devices, said users could back up their data to its own cloud without trusting the company with the key.

[…]

The company continues to offer the service but declined to comment on how many users have taken up the option.

John Gruber:

Apple, the privacy company

Ryan Jones:

We all need to remember and repeat often: “Apple has a master key for all iCloud backups”.

Matthew Green:

Notice as well that Apple didn’t just abandon plans to encrypt iCloud backups by default. They even abandoned giving users the option to encrypt those backups. And due to the walled garden nature of iOS, there’s nothing you can do about it.

Benjamin Mayo:

If Apple wants to be taken seriously as the ‘privacy’ company, then they should offer end-to-end encrypted iCloud backups.

Hide it behind ten warning dialogs, whatever. It should be an option, plain and simple.

The iOS security model (rightly) won’t let any other company have privileged access to the complete filesystem and user data, so a third-party solution isn’t possible. It falls on Apple to do it.

Matthew Green:

I was going to write something about iCloud backup and encryption but I realize that I’ve written it all. So here are a bunch of old posts[…]

Walt Mossberg:

As most of you know, you can back up an iPhone or iPad locally, to your own computer, using Apple’s iTunes program. In fact, that was the only way to back up your iPhone before Apple introduced iCloud Backup in iOS 5 in 2011. It was originally introduced with the iPod many years before.

This method is clumsy, slow, and requires you to remember to use it. But it still works. You can even opt to encrypt this local backup. And it keeps the contents of your phone available for restoring, but out of the cloud and out of Apple’s control.

Royce Ausburn:

It’s interesting that they choose to make iPhones unbreakable but not the backups. I’d prefer it the other way around given a choice. Better for law enforcement too, they get iCloud backups without the suspect’s knowledge, but the suspect would know if their iPhone is seized.

Eric Young:

I think this entire debate is purposefully vague and abstract

Apple doesn’t want to and cannot take firm stances on “privacy”

Because they can’t. Their statements are purposefully vague and abstract - and what many will come realize: misdirection/and somewhat dishonest

[…]

The idea that a for-profit company jumped into this un-winnable and resource draining war - is shocking to me

This is Apple’s war on drugs. It’s their lost decade. And we all suffer because of it

I’m not sure it’s true that Apple’s privacy stance is the reason for their poor services, but fighting governments on encryption is difficult. At present, Apple seems focused on putting up a smokescreen so that customers think their data is more secure than it actually is. The way to win, if that’s even possible, would be to educate the public about what’s actually going on, so that they can lobby their representatives to change government policy in favor of privacy.

Previously:

Update (2020-01-21): AAPL of Discord:

Steve Jobs responding to Walt Mossberg on Privacy at D8 2010:

“No! Privacy means that people know what they’re signing up for. In plain English, and repeatedly. That’s what it means.”

At the very least, Apple is failing on the “plain English” bit.

zacwest:

You need to be extremely technical to understand the difference between “Encryption: Yes” and not end-to-end encrypted. To the lay user, Apple is explicitly telling you that they’re encrypted.

And to understand that “Messages in iCloud also uses end-to-end encryption” actually means that Apple can access the messages if they are backed up and could access future messages without a backup if they added a fake device to your account.

David Sparks:

So Apple is holding the line on our devices but not on our backups. That seems like a great way to upset everyone. Do they think giving the government user just iCloud backups will satisfy them? Do they think that privacy-minded users will say “good enough” when they realize their device is encrypted but not their backups? Seems to me like it is time for Apple to fish or cut bait.

Nick Heer:

Even though Apple attempts to explain how iCloud backups work, I don’t think they do a good job, and it is one reason the Reuters report today had such a profound impact: a lot of people have been surprised that their iCloud backups are less private than their phone. Yet, as bad as this is for Apple, it is equally a poor look for the Department of Justice, who have publicly been whining about their inability to extract device data while privately accepting Apple’s cooperation.

John Gruber (tweet):

It’s essential that Apple still supports local backups, for many reasons, but for most iPhone and iPad users it’s irrelevant, because they never connect their devices to a Mac or PC, and the overwhelming majority of them surely have no idea that the feature even exists. iCloud backups are the only backups most iOS users ever use, and it is a fact that there is no option to truly encrypt them.

[…]

In fact, it’s so contrary to Apple’s stance as The Privacy Company that I’ve already heard from several tech-savvy users today, in the wake of Reuters’s report, that they had assumed until now that their iCloud backups were encrypted.

[…]

If that is the case — that Apple’s legal department killed the project to avoid “poking the bear” — then it’s ultimately irrelevant whether Apple briefed the FBI in advance or not. It’s acquiescence, and users will be left unprotected. Not just in the U.S., where the FBI has jurisdiction, but everywhere in the world where encryption is legal.

Apple Legal is afraid to poke the bear, which Google has been doing since 2018?

Starting in Android Pie, devices can take advantage of a new capability where backed-up application data can only be decrypted by a key that is randomly generated at the client. This decryption key is encrypted using the user’s lockscreen PIN/pattern/passcode, which isn’t known by Google. Then, this passcode-protected key material is encrypted to a Titan security chip on our datacenter floor. The Titan chip is configured to only release the backup decryption key when presented with a correct claim derived from the user’s passcode. Because the Titan chip must authorize every access to the decryption key, it can permanently block access after too many incorrect attempts at guessing the user’s passcode, thus mitigating brute force attacks. The limited number of incorrect attempts is strictly enforced by a custom Titan firmware that cannot be updated without erasing the contents of the chip. By design, this means that no one (including Google) can access a user’s backed-up application data without specifically knowing their passcode.

Ryan Jones:

We must stop using “encrypted” and especially “end-to-end encrypted”.

ICloud backups are encrypted, but Apple has a master key.

All that matters is accessible. Are iCloud backups accessible by Apple? Yes.

Why does Apple have a master key? Because people forget their passwords every day.

I believe that’s a valid reason and why they have access (not for some FBI conspiracy).

The problem is Apple hides it.

You have to read this like a detective to know.

Not just backups, but also the Safari browsing history for all your devices.

Monday, January 20, 2020 [Tweets] [Favorites]

Creating Root-level Directories and Symbolic Links in Catalina

Rich Trouton:

The use of firmlinks is exclusively reserved for the OS’s own use, but Apple has also made available what are called synthetic firmlinks. These synthetic firmlinks are how the OS enables folks to create directories and symbolic links on the read-only boot volume.

[…]

Whomever designed this came down on the “tabs” side of the “tabs vs. spaces” debate. When creating the separation between installers and Users/Shared/installers in the /etc/synthetic.conf file, you need to use tabs. If you use spaces instead, the synthetic firmlink won’t be created.

[…]

For more information, please see the synthetic.conf man page.

Previously:

Farewell, Carcassonne

The Coding Monkeys (tweet):

With a heavy heart we have to announce that our rendition of Carcassonne will no longer be available for purchase as of March 1st 2020. Our contract with the Company behind Carcassonne, Hans im Glück, has come to an end. Shortly after that, the interpretation by Asmodee will be available for purchase for Mac and iPhone/iPad, so your Carcassonne needs will be catered for in the future.

[…]

This has been an amazing ride for us. Our online service catered for over 2 Million active devices at peak times. You finished over 8.5 Million games, and on average there are around 3.500 games up and running at any point in time. We launched on non-retina iPhones, and expanded and supported over 20 models of iPhone and 19 models of iPads. From iPhone OS 3 up to iOS 13 today. We proudly have a service uptime of over 99,999% during that decade, thanks to our erlang-based technology.

Previously:

Profiling Mac Unit Tests

Ilja A. Iwas:

Mac Devs: Is it just me, or is the “Profile ‘Test XYZ’” command in Xcode 11.3 broken? Instruments launches the main app, but does not run any tests. 🤷‍♂️

He’s filed bug FB7543911.

There was a brief period of time where running tests under Instruments worked for me, and it was glorious. Then the handy “Profile [test]” command stopped working—Instruments would launch but not record anything from the test. I reverted to the old method of setting a breakpoint and manually attaching Instruments to the test process while it was paused. Lately, that no longer works for me, either.

Mostly, I use Instruments to track down memory leaks. An alternative tool is the Memory Graph Debugger, which works within Xcode itself and never seems to have trouble attaching/recording.

Rejected for Working Around macOS Bugs

Daniel Jalkut:

It’s a dramatic day for @MarsEdit and compatiblity, as App Review has suddenly become more interested in private WebKit SPI. I patch WebHTMLView to work around serious bugs which I have filed. One of them is a crasher. I don’t think WebKit1 is the best focus for App Review?

A truly unfortunate situation to be in as a developer. Apple will likely never fix the bugs, as WebView is now deprecated. Its replacement, WKWebView, is not fully ready yet, and will require a complete rewrite in a different language to get the same functionality, if that’s even possible.

Daniel Jalkut:

Spending my whole weekend, apart from celebrating the delightful birthday of my little baby 8yo, working around issues that App Store review put on my plate. The price I pay for playing this game, I know. I just wish the game were a little different.

[…]

Overall I am pretty chill about App Store review and the goals of bringing developers into line. I do think it would be massively improved by a graduated system of warning of future rejection while allowing immediate fixes to pass through.

As a Mac App Store developer whose apps have been in the store since the beginning, it’s not a great feeling to know that any critical update might be held up because Apple decided to get more uptight about something that was OK for the past 8 years.

Daniel Jalkut:

Generally I would say the thing for other developers to look out for is Apple may be improving its ability to detect things like patches as opposed to outright “use” of private API, and they may also be getting less forgiving of some behaviors, even if in the name of better UX.

This is the thanks you get for filing Radars and putting up with bugs for all those years.

It’s not exactly altering the deal because the Mac App Store has always banned private API. As was said at the outset, this is not realistic. There will always be (different) OS bugs. Even in the best case, they will take time to fix. Nobody—not customers, developers, nor Apple—wants apps to exhibit bugs, but that’s the inevitable result of a policy that forbids patching to work around buggy API.

When the Mac App Store debuted with this policy, some people said it would force Apple to fix the bugs faster. I don’t think that’s happened. Rather, developers kept doing what they were doing—I bet most large or popular apps are using private API to work around bugs—and Apple either failed to detect this or chose to look the other way.

Now, the rules haven’t changed, but perhaps enforcement has. This is a problem both because of increased user-visible bugs and fairness. Some apps like MarsEdit will eat up development time to end up with something buggier than they started with. Other apps will get a different reviewer and slide right through. Apps that Apple deems sufficiently important will be exempt from the rule.

With out-of-process Web views, increasing use of Swift, and direct Objective-C properties, patching will be more difficult. This will level the playing field—but to the lowest common denominator.

Daniel Jalkut:

My compromise build was approved by Apple this afternoon. I found another way around one of the bugs I was fixing, but have no fix for the other one, yet. It’s a fairly minor thing, but the Mac App Store version is now buggier than it was, thanks to App Review.

Previously:

Update (2020-01-20): Jeff Johnson:

Apple currently lets Catalyst apps use private API as workarounds, but at some unpredictable time they’ll get rejected for it.

Friday, January 17, 2020 [Tweets] [Favorites]

The Flask Mega-Tutorial

Miguel Grinberg:

Welcome! You are about to start on a journey to learn how to create web applications with Python and the Flask framework. The video above will give you an overview of the contents of this tutorial. In this first chapter, you are going to learn how to set up a Flask project. By the end of this chapter you are going to have a simple Flask web application running on your computer!

Via Sean Heber:

[It] is quite possibly the best and most complete technical tutorial I’ve ever come across.

Malicious WebAssembly

Catalin Cimpanu:

In an academic research project that was carried out last year, four researchers from the Technical University in Braunschweig, Germany, looked at WebAssembly’s use on the Alexa Top 1 Million popular sites on the internet, in an attempt to gauge the popularity of this new technology.

[…]

However, while the vast majority of samples were used for legitimate purposes, two categories of Wasm code stood out as inherently malicious.

The first category was WebAssembly code used for cryptocurrency-mining. These types of Wasm modules were often found on hacked sites, part of so-called cryptojacking (drive-by mining) attacks.

The second category referred to WebAssembly code packed inside obfuscated Wasm modules that intentionally hid their content. These modules, the research team said, were found part of malvertising campaigns.

Via Kyle Howells:

WebAssembly should be a user opt-in per website feature.

App Movement Monitoring

Daniel Jalkut:

Essentially, when a Mac app is launched, the location of that app on disk is saved, and used repeatedly whenever an internal component needs to located. The dynamic nature of resource loading in Mac apps means that these components are not typically loaded until they are need. For example, if you never show the Preferences window in a typical Mac app, the resources that define that window will never be loaded.

If, on the other hand, you decide to show the Preferences window, but you’ve moved the app since it was launched, things have a tendency to go haywire. The app will go searching for its Preferences resources in the location on disk where they used to be, and it won’t find anything.

Why doesn’t Cocoa use file reference URLs to delay resolving the path until it’s actually needed, like it does for the document architecture? Or at least try to prevent users from moving apps that are open?

Paul Kafasis:

To avoid these issues, the Finder already works to avoid multiple types of changes to running applications. For instance, if you try to delete an application that’s open, the Finder stops you[…] As well, if you attempt to rename an open application, the Finder will warn against it[…] However, while any Unix geek can tell you that a rename is really just a move by another name, the Finder does nothing to stop you from actually moving the app.

Paul Kafasis:

I discussed this issue with Daniel and Brent, and provided them with the code we’d been using to watch for this issue. This actually led us to make several changes and tweaks, and a simplified implementation of this “Application Moved” watcher can be found below[…]

RSAppMovementMonitor:

RSAppMovementMonitor handles the nuanced details of monitoring for the movement, prompting the user, and relaunching the app[…]

This is more complicated than it seems, due to quarantine, translocation, sandboxing, and need to preload any localized strings before the app is moved.

The latter was surprising to me, but I recently ran into a similar issue when updating my apps to en.lproj instead of English.lproj.

Previously:

Thursday, January 16, 2020 [Tweets] [Favorites]

YouTube RSS Feeds

Thomas Brand:

In 2020 I am watching less stupid on YouTube by skipping the algorithm. Instead of letting the YouTube decide which videos it wants to show me, I am watching only the videos I want to see by subscribing to my favorite content creators via RSS.

[…]

[Append] the Channel ID to the end of this URL https://www.youtube.com/feeds/videos.xml?channel_id=

The combined URL + Channel ID is the Feed URL of that YouTube channel, and can be added to my feed reader.

Too bad there’s no way to do this for Twitter.

Nick Heer:

YouTube isn’t the only website that buries its RSS feeds in this manner. I don’t know that it’s deliberate — in the sense that they’re trying to discourage the use of RSS. I think it might be a result of product teams convincing themselves that RSS is something used only by the technically-proficient, so it’s put in a place where that group can find it. The trouble is that only the technically-proficient will end up using it, so it’s cyclical.

Why not just put it in a <link> tag?

Previously:

Optionals in Swift Objective-C Interoperability

Fabián Cañas (tweet):

The problem is that since Swift doesn’t think this value can be nil, it’s not trivial to check.

[…]

It says the non-optional value shouldn’t be compared to nil, and that it’s always false. But at run time, the nil is detected, and we print the statement.

[…]

What’s interesting here is that the argument to the bridge function is an Optional<NSCalendar>. The static method, by its signature, accepts nil. What’s happening then? In this case, The culprit for the crash and what saves us from unexpected behavior later on is a force unwrap. Though the value that’s actually passed in to the function is Optional<NSCalendar>.some(nil), which is still not a valid value and we’re still in undefined behavior territory, so it’s pleasantly surprising that a force unwrap catches this case.

[…]

Having the the compiler automatically check and assert that nonnull Objective-C types returned by Objective-C methods are indeed present would be fantastic, whether for debug builds or as an independent flag.

Brent Royal-Gordon:

To make sure we’re all on the same page: returning null from a nonnull imported API is full-on, demons-flying-out-of-your-nose undefined behavior. There’s no guarantee that it will do what you saw.

Unfortunately, it’s rather easy to get the annotations wrong, and even Apple does this. For example, the SecDigestTransformCreate() and SecTransformExecute() calls can return NULL in Objective-C, but Swift acts as if they can’t fail. I filed a bug about about this, which Apple recently said was so old that they wanted to close it and have me open a new one. Meanwhile, I’m able the work around the issue because these two APIs have a separate error pointer that can be examined. Without that, I think you would need an Objective-C wrapper to safely detect whether an error has occurred.

Is Git Irreplaceable?

wyoung (Hacker News):

I worry that Git might be the last mass-market DVCS within my lifetime. Git effectively has a global monopoly on DVCSes, and I don’t see how you replace such a thing.

Replacing RCS with CVS was easy. Replacing CVS with Subversion was a big fight in many places. Replacing the remaining CVS and Subversion repos with something modern may never happen. Replacing Git with something better looks impossible.

Via Greg Hurrell:

My take: scalability is the biggest hurdle (weak spot) that must be overcome; complaints about usability are majorly exaggerated – version control has some inherent complexity that can’t be elided.

I think that’s right. The command-line weirdness is gradually being addressed, and you can mostly avoid it by using a GUI, anyway. The alternatives I’ve seen are also inherently complex.

Previously:

Composite Mac Desktop Picture

J3nRa1n:

No one asked for this. But here it is: every macOS wallpaper from Mac OS X 10.0 Cheetah to macOS 10.15 Catalina combined.

The full-resolution image is here, and there’s also one for iOS.

Previously:

Which Emoji Scissors Close?

wh0 (via Nick Heer):

Conveniently, the emojis studied in this post depict the scissors from a viewpoint parallel to the axis of the hinge. This allows us to simulate swinging the blades with basic image rotations. I collected a dataset of emojis from different vendors from Emojipedia. In the following experiments, I swing the blades around the hinge until the handles collide with each other.

Wednesday, January 15, 2020 [Tweets] [Favorites]

AppleScript to Export Open Safari Tabs to OmniFocus

Jesse Squires:

I am often in a situation where I have a number of tabs open in Safari. I may be reading a collection of blog posts about how to implement a new iOS API, or I may be researching something I need, like new running shoes. I cannot always complete the task in that moment and I want to revisit it another time, or I want to save all the links for later. If they stay in Safari (even as bookmarks) they will be lost forever to me. I need to save them into OmniFocus. So I wrote an AppleScript to do that.

Typewriter Keylogger

Kyle Mizokami (via Espionage News):

The NSA eventually shipped all of the electronics located at the embassy back to the U.S. for study. They struck gold: parts inside an IBM Selectric typewriter had been cleverly duplicated but rigged to transmit the typist’s keystrokes. The typewriter still worked, but it also quietly broadcast the keystrokes, using Soviet over-the-air TV signals as a form of electronic camouflage. It was in effect a non-digital form of the keylogging malware that hackers install on PCs.

Robert W. Lucky:

A solid aluminum bar, part of the structural support of the typewriter, had been replaced with one that looked identical but was hollow. Inside the cavity was a circuit board and six magnetometers. The magnetometers sensed movements of tiny magnets that had been embedded in the transposers that moved the typing “golf ball” into position for striking a given letter.

Other components of the typewriters, such as springs and screws, had been repurposed to deliver power to the hidden circuits and to act as antennas. Keystroke information was stored and sent in encrypted burst transmissions that hopped across multiple frequencies.

Previously:

Why Wireless Networks Pose No Known Health Risk

Glenn Fleishman:

More recently, the Chicago Tribune published the results of testing from a firm it had hired to check if emissions from modern smartphones truly fell below FCC safety limits. In those tests, many appeared to exceed regulatory limits. The Tribune didn’t overstate its results, but the bottom line was, more or less, positing that smartphone makers were all deceiving the FCC and the general public. This plays into our fears, even though the work was presented rigorously. (Smartphone makers dispute the methodology of the testing; the Tribune stands by its research. Regardless, there’s a big difference between detecting higher-than-approved emission levels and proving a link between those levels and cancer.)

[…]

To achieve the promised high rates of speed and serve new categories of devices, 5G networks will draw from a much broader range of frequencies, some far higher (or shorter) wavelengths than current technologies. And many times as many base stations will need to be deployed.

But the newness and differentness of 5G don’t matter. Whether we’re talking about 5G, 4G, 3G, Wi-Fi, or other consumer-level wireless technologies, the sum total of results from many studies and many years of research paints a straightforward picture—there’s nothing to worry about.

Bluetooth LE Audio

Bluetooth SIG (MacRumors, Hacker News):

As the names suggest, Classic Audio operates on the Bluetooth Classic radio while LE Audio operates on the Bluetooth Low Energy radio.

[…]

LE Audio will include a new high-quality, low-power audio codec, the Low Complexity Communications Codec (LC3). Providing high quality even at low data rates, LC3 will bring tremendous flexibility to developers, allowing them to make better design tradeoffs between key product attributes such as audio quality and power consumption.

[…]

LE Audio will enable the development of Bluetooth hearing aids that bring all the benefits of Bluetooth audio to the growing number of people with hearing loss.

[…]

LE Audio will also add Broadcast Audio, enabling an audio source device to broadcast one or more audio streams to an unlimited number of audio sink devices. Broadcast Audio opens significant new opportunities for innovation, including the enablement of a new Bluetooth use case, Audio Sharing.

Andrew Liszewski (via John Gordon):

[One] of the biggest improvements it will include will be a feature called Multi-Stream Audio. Bluetooth is currently limited to streaming audio to just a single device. That’s fine for portable speakers and headphones where both sides are connected with a wire, but for wireless earbuds, such as Apple’s AirPods, your smartphone can actually only connect to one side. That earbud then has to forward the audio stream onto the one in your other ear, which requires some clever software tricks to ensure everything remains in sync.

Tuesday, January 14, 2020 [Tweets] [Favorites]

Low Power Mode for Mac Laptops

Marco Arment:

In light of today’s rumor that a Pro Mode may be coming that seems to offer benefits in the opposite direction, I wanted to re-make the case for a Low Power Mode on macOS — and explain why now is the time.

[…]

Apple’s customers don’t usually have control over these balances, and they’re usually fixed at design time with little opportunity to adapt to changing circumstances or customer priorities.

[…]

Turbo Boost Switcher Pro relies on a kernel extension that’s grandfathered into Apple’s latest security requirements, but it can never be updated — and when macOS Catalina loads it for the first time, it warns that it’ll be “incompatible with a future version of macOS.” I suspect that this is the last year I’ll get to run the latest OS and be able to turn off Turbo Boost at will, making all of my future laptop usage significantly worse.

Previously:

The Security of Safari Extensions

Jeff Johnson:

Every Safari extension that runs JavaScript — in other words, almost every Safari extension — will have these same warnings, so what are users supposed to do with the warnings? Avoid Safari extensions entirely? Then why does Apple provide a developer API for Safari extensions, if they aren’t meant to be used? Why are Safari extensions available in the Mac App Store? Why does Apple advertise that they’re available in the Mac App Store? You get the feeling that different teams within Apple are not on the same page here, and they’re giving unhelpful mixed messages to users.

[…]

In general, my view is that you shouldn’t install software on your Mac unless you trust the developer. You can’t rely on the system to protect you from malicious software, because there are always vulnerabilities and ways to get around the system.

It’s a recurring theme. Without the scary warning, Apple gets blamed for any problems. With the warning, developers blame Apple for scaring customers away from their products, and if anything bad happens everyone blames the customer for ignoring the warning that they had no way to evaluate.

Previously:

Aerial Screensaver and Catalina

Guillaume Louel (via Tanner Bennett):

In macOS Catalina, 3rd party screensavers are now running in a sandboxed container which limits everything. As of right now there are no workarounds for many of the restrictions.

[…]

Aerial can only write in the legacyScreenSaver.appex sandbox container, which means in Catalina, JSON files and videos can only be downloaded in ~/Library/Containers/com.apple.ScreenSaver.Engine.legacyScreenSaver/Data/Library/Application Support/Aerial. Aerial can still read (only) the rest of your system disk so you can still store the videos in another folder after they are downloaded by manually setting the Cache. But Aerial cannot download videos to this cache.

[…]

Your cache may be wiped by the Catalina installer

[…]

[Because] a screen saver is not an app (we are a plugin run by legacyScreenSaver) we can’t ask for entitlements for, say, accessing filesystem.

[…]

Some (not all) of Apple screensavers are now bundled as an .appex too, with their own permissions. As far as I know that format is still not documented to this day nor available to 3rd parties.

Previously:

The Dark Side of Dark Mode and Night Shift

Adam Engst (tweet, Hacker News):

Unfortunately, Apple’s marketing claims about Dark Mode’s benefits fly in the face of the science of human visual perception. Except in extraordinary situations, Dark Mode is not easy on the eyes, in any way. The human eyes and brain prefer dark-on-light, and reversing that forces them to work harder to read text, parse controls, and comprehend what you’re seeing.

[…]

In the scientific literature, black on white is called “positive polarity,” whereas white on black is called “negative polarity.” Numerous studies over decades of research have found that positive polarity displays provide improved performance in a variety of areas.

[…]

When there’s a mismatch between the two—the screen is too dim outside or too bright inside—it’s hard to look at. That’s why Apple implemented automatic brightness control in iOS (find it in Settings > General > Accessibility > Display Accommodations) to reduce the screen brightness when you’re reading in a dark bedroom and increase it when you’re trying to take a picture on a sunny day.

I haven’t personally found any use for Dark Mode on my Mac. I don’t like the way it looks, and it feels like it slows me down. I have always preferred light text on a dark background for code, though. My sense is that this is not because I like light-on-dark better than dark-on-light for the primary text, but rather because most of the other colors work better on a dark background. With multi-color themes, the secondary colors tend to be easier to see on a dark background.

I don’t like how Dark Mode looks on iOS, either, except that I’ve always preferred Tweetbot in dark. I also like to run OmniFocus in dark so that the screen isn’t so bright if I’m making a note at night. Automatic brightness control just doesn’t cut it.

I continue to like Night Shift.

Jonathan Wight:

Giving on up dark mode.

Don’t like.

I get double vision due to my eye condition on most text and in almost all dark mode implementations it’s far more pronounced.

John Gruber found that Dark Mode helps with his eye condition.

Chance Miller:

These features are designed to change the temperature and color of your display based on what time of day it is. New research suggests, however, that features designed to reduce blue light before bedtime might not be as effective as initially thought.

Previously:

Monday, January 13, 2020 [Tweets] [Favorites]

macOS Filename Homoglyphs Revisited

Adam Chester (tweet):

Last year I posted a few tricks to help when targeting MacOS users, and included a technique useful for spoofing file extensions with the aim of taking advantage of Finder’s removal of the .app extension from certain filenames.

A few weeks ago I was about to use this on an engagement and found that Apple had patched the tricks presented previously. While this was frustrating for me as an attacker, it did provide an opportunity to dig into the fix, understand just how filenames are now being sanitised by MacOS Catalina, and see if I could bypass this again.

See also: Howard Oakley.

Lambda the Ultimate Pattern Factory

thma (via Heath Matlock):

One of my earliest findings was that several of the GoF-Patterns had a stark resemblance of structures that are built into in functional languages: for instance the strategy pattern corresponds to higher order functions in fp[…]

Recently, while re-reading through the Typeclassopedia I thought it would be a good exercise to map the structure of software design-patterns to the concepts found in the Haskell type class library and in functional programming in general.

iPod and Other History via Tony Fadell

Tony Fadell:

It took 4+ yrs to ship Magic Cap v1.0 - our leaders wanted to avoid making the same mistakes they had made with MacOS v1.0 We all learned how important it is to stick to a schedule. The “new” internet crushed GM. Real artists ship - fast & iterate fast!

Tony Fadell:

The real reason why iPod & iPhone were successful was based on the market timing & the complete ground up design (HW+SW) That said, I pushed to ship the 1st iPod in <10 months. Long timelines are the death of daring projects inside (struggling) companies. We even surprised SJ!

Tony Fadell:

[The small Toshiba hard drive] was a critical component no doubt. But even more important was the “exclusive supply” agreement I/we negotiated to enable us to get the hdd for 3+ years before any of our competitors could… Toshiba didn’t think MP3 players were a big market - they thought it was laptops.

Benedict Evans:

At 3GSM in 2005 a Motorola exec explained to me how hard they were working to put an HDD into a phone to compete with you. Six months later the Nano came out...

Tony Fadell:

I bet the whole product & business on a processor from a tiny unknown startup who had failed once to deliver. Most big companies would NEVER do such a risky thing.

Tony Fadell:

Design + HW + SW + App (then later + Content)

But even more importantly - continually introducing new products at lower price points while adding new features at the high end. That was the way iPod locked up the market with consumers.

Tony Fadell:

Many sleepless nights worrying about Sony. It was one of my biggest questions I asked Steve during the first pitch “Sony owns every audio market category.…”

MSFT - no worries - they weren’t a SW+HW innovator like they are today. I would worry about them now however.

Tony Fadell:

I was at the starting point of first 18 gens of iPod that shipped (& a few generations that didn’t ship). Was a team effort, but mainly technology driven since we could only deliver what we could push the tech to do.

You are defined by what you do & also by what you don’t do.

Tony Fadell:

I remember the day when Steve called me to the Board Room to personally sign a $4B purchase order for Samsung Flash for the Nano. “Are you sure we are ordering the right stuff? It’s going to work, right?” It was the biggest single order Apple had ever placed at the time.

Tony Fadell:

iPod Nano, yet-to-be-launched, was about to use >40% of the world’s flash. Samsung would build new factories to support our, what we hoped, incredible sales volume.

Tony Fadell:

Sony Corp was too concerned about the declining CD music sales in the Sony Music Entertainment business. They didn’t want to appear to embrace MP3s - since that implied the music was stolen. Corporate politics. Reminds me of Kodak who invented the digital camera…

Tony Fadell:

Luckily our good friend @waltmossberg helped us (or should I say “convinced the non-believer, Steve”) make the (highly religious) decision ship the iPod for PC!

Walt Mossberg:

Just to be clear: I had no agenda, owned no Apple or Microsoft stock and took no money or goods from either. As I recall, Jobs and I were in the middle of a long chat, and he asked what I thought of putting iTunes on Windows. I asked if he was considering it. He said yes.

I merely said two true things: (a) some readers had been emailing me to ask if this was going to happen and (b) there was a low upper limit to Mac owners. I wasn’t trying to “convince” him and left thinking only that I’d learned, off the record, that he was considering it.

Steve Jarrett:

We started working on Windows sync earlier, right after xpod. There even was a secret iPod Windows Connectivity SDK for any app to sync. MusicMatch used that. Steve changed his mind, killed the SDK, and decided instead to port iTunes to Windows. Fateful.

Walt Mossberg:

I recall him actually asking me if MusicMatch was a good Windows app. I said yes and pointed out that I had given it favorable reviews in my column. I had the impression he was going to sort of designate it as the Windows software for the iPod.

Tony Fadell:

This was the exact conversation I referred to & that @waltmossberg didn’t know about. Afterwards we committed to shipping iPod on the PC… the rest is history.

This was Steve’s way of settling this highly contentious decision. He didn’t want to say YES to us & we wouldn’t take NO from him. So SJ said, well let’s get someone external, who we trust, to help us get to the right answer…

Triangulation:

What do you get when the creators of the Macintosh, iPod, Android, Ebay, Linked In, Nest, Flash, and the future CTO of the United States try to invent the iPhone a decade before its time? General Magic was a spin-off of Apple tasked with creating the ultimate personal digital assistant. It failed. Filmmakers Matt Maude and Sarah Kerruish join Leo Laporte to talk about their new documentary, General Magic, which chronicles the rise and fall of the greatest failed company in Silicon Valley.

See also: Tony Fadell on The Tim Ferriss Show (tweet).

Previously:

Friday, January 10, 2020 [Tweets] [Favorites]

Project Zero Remote iPhone Exploitation

Samuel Groß (Hacker News):

This is the first blog post in a three-part series that will detail how a vulnerability in iMessage can be exploited remotely without any user interaction on iOS 12.4 (fixed in iOS 12.4.1 in August 2019).

[…]

This research was mainly motivated by the following question: given only a remote memory corruption vulnerability, is it possible to achieve remote code execution on an iPhone without further vulnerabilities and without any form of user interaction? This blog post series shows that this is in fact possible.

[…]

For the purpose of this blog post series, it is important to realize that a vulnerability in the NSKeyedUnarchiver API can generally be triggered in two different contexts: in the sandboxed imagent and in the unsandboxed SpringBoard process (which manages the main iOS UI, including the homescreen).

Samuel Groß (Hacker News):

The initial primitive gained from the vulnerability is an absolute address dereference in which the read value is afterwards used as an ObjC object. As such, some knowledge of the target address space is required in order to exploit this vulnerability for remote code execution. This blog post describes a way to defeat ASLR remotely without any additional information disclosure vulnerabilities.

First off, the effectiveness of an old technique, heap spraying, is evaluated. Afterwards, a technique is described through which it is possible to infer the base address of the dyld shared cache region given only a memory corruption bug. The released code implements the presented attack and can infer the shared cache base address remotely on vulnerable devices within a couple of minutes.

Samuel Groß (Hacker News):

At this point, ASLR has been broken as the shared cache’s base address is known and controlled data can be placed at a known address with the heap spray. What remains is to exploit the vulnerability one more time to gain code execution.

After a short introduction to some relevant ObjC internals, an exploit for devices without pointer authentication (PAC) will be outlined. It involves creating code pointers, so it no longer works with pointer authentication enabled. Afterwards, a different exploit that works against PAC and non-PAC devices will be presented. Finally, a technique to chain the presented attack with a kernel exploit, which involves implementing the kernel exploit in JavaScript, will be shown.

Previously:

Observing Appearance Changes

Jesse Squires:

I needed to get notified when the system appearance changed. On iOS, this is very straight-forward and a first-class API. On iOS 13, the interface style is part of UITraitCollection.

[…]

[On macOS,] you can KVO on NSApp.effectiveappearance. […] Using KVO still does not feel great, but this is the best solution I could find without having an explicit API like iOS.

Previously:

KVO, My Enemy

Brent Simmons:

One of the keys to the stability of the shipping versions of NetNewsWire is that we don’t allow KVO (Key-Value Observing).

KVO is a false convenience — it’s often easier than setting up a delegate or old-fashioned notification. But to use KVO is to just ask for your app to crash.

And not just crash, but crash in hard-to-figure-out ways.

Drew McCormack:

As others point out, there are a bunch of issues with the FeedlyOperation class, such as the isAsynchronous override, and not changing to the final state atomically. But I think maybe the biggest problem is the use of Swift key paths. Not sure if that should be supported, but in my testing, it doesn’t work. Changing to strings for the key paths fixed things for me.

I’m not sure what the issue is—perhaps a Swift bug or incorrect property declarations so that there’s disagreement about whether is is part of the property name—but Swift key paths seem to be a common source of problems.

Brent Simmons:

We’re going to write a replacement for OperationQueue.

Previously:

How to Downgrade a New Mac to Mojave From Catalina

Armin Briegel:

Apple has started shipping Mac models that used to come with Mojave pre-installed with Catalina. If your organization has blockers for Catalina (incompatible software, etc.) you may want to install Mojave on these Macs. Unfortunately, this is not so easy.

[…]

Directly downgrading from Catalina to Mojave with the startosinstall --eraseinstall command will fail. Attempts to run the Mojave installer from a Catalina Recovery (local or Internet) will also fail. The reason seems to be that the Mojave Installer application chokes on some aspect of Catalina APFS.

[…]

The solution requires a Mojave Installer USB disk.

[…]

To boot a new Mac with a T2 chip off an external drive, you need to allow external boot from the Security Utility in the Recovery partition.

Via Josh Centers:

We’re still hesitant to recommend upgrading to Catalina, particularly if you use Mail, since reports of email data loss continue.

I continue to receive reports of Mail messages lost when upgrading/migrating/rebuilding and when moving messages between mailboxes (particularly between different accounts). There also seem to be a bunch of new crashing bugs affecting Exchange users, as well as problems with messages not moving (but, thankfully, not disappearing either). Yet many customers are also using Catalina without these issues. It’s not yet clear why some are affected and others aren’t, but it’s good to know that downgrading is possible (unless you have a 16-inch MacBook Pro or a 2019 Mac Pro).

Previously:

Thursday, January 9, 2020 [Tweets] [Favorites]

End & Accept, Decline, Hold & Accept

Brenden Mulligan, on the three choices given when you’re on an iPhone call and a new call comes in:

These options break my brain every time.

iOS 11

The issues:

1. It’s unnecessary decision making in the 10 seconds before the new call goes to voicemail

2. It’s too specific about what’s going to happen to the current call

[…]

This is an interesting one because you don’t see this in a stress free state. You see it while juggling at least one other task, and you have time pressures, etc. and you don’t see this UI very often.

I find this confusing, too. One issue is that the language isn’t parallel. In two of the choices, the first word applies to the current call, but in the other one it applies to the new call. Keeping the current structure, I think it would be clearer to have either:

I also find the icons too hard to interpret quickly. One issue is that the red circle with the hung-up phone doesn’t mean the same thing in both cases.

Shekhar Somani:

[This] screen breaks the muscle memory. We don’t panic when the first call arrives because there are two clear options (if the phone is unlocked) – Accept on the right and Decline on the left.

Whereas, now Decline is in the middle.

John Gruber:

[It’s] the same exact location as the big red end button that does end the current call before (and after) the incoming call arrives.

Willi Kampmann:

The big red phone button in the middle is deeply encoded into my brain as “end call”. I’d rather see a big X or sth to signify declining another call.

John Gruber:

Took me a while to find it, but I knew in my gut that Apple had completely knocked this one out of the park — in 2007.

iOS 1

[…]

Watching and listening to the incoming call in the video, it is even more clear. You have a thing — the current call. The current call has an End Call button. A new thing arrives — the incoming call. Two options for new thing that don’t end the existing thing are grouped.

This is so much better. No confusing icons, and the red button consistently means to end the current call.

Dark Patterns on Travel Websites

Chris Baraniuk:

Being a cyber-security researcher, she was familiar with web code so she decided to examine how OneTravel displayed its web pages. (Anyone can do this by using the “inspect” function on web browsers like Firefox and Chrome.) After a little bit of digging she made a startling discovery – the number wasn’t genuine. The OneTravel web page she was browsing was simply designed to claim that between 28 and 45 people were viewing a flight at any given moment. The exact figure was chosen at random.

Via Nick Heer:

Also, many of the biggest travel booking websites are owned by just a couple of companies: Bookings Holdings runs Booking.com, Priceline, Kayak, and Cheapflights; the Expedia Group owns Expedia, Hotels.com, Hotwire, Orbitz, Travelocity, and Trivago. Each group shares the same inventory, and they all use the same tactics. Users simultaneously get the impression that they’re shopping around and competing with other users, when neither is true.

Front and Center 1.0.1

John Siracusa (tweet):

In classic, when you click on a window that belongs to an application that’s not currently active, all the windows that belong to that application come to the front. In Mac OS X (and macOS), only the window that you click comes to the front.

[…]

Sadly, macOS Catalina’s lack of support for 32-bit apps finally killed the last of the apps that implemented this feature. I was alone in a cold, barren world where I had to click on a Dock icon to switch to an app and bring all its windows to the front.

His Front and Center app lets you choose the classic behavior or, as I prefer, choose the modern behavior and selectively override it by holding down the Shift key when you do want all the windows. There are ways to do this without the app:

But a modified click is more elegant.

Lee Fyock:

Gus knew of a deprecated API that does the process-switching much more efficiently, that doesn’t exhibit the same bug, and makes the code much simpler. Given that the impetus of writing the app was to make the 32-bit to 64-bit transition cleanly, I wasn’t a fan of using an API that had been deprecated in OS X 10.9, but it works well.

Carbon for the win. I, too, have had issues with the newer process APIs.

Previously:

Update (2020-01-10): John Gruber:

So why Shift-click? There really wasn’t any choice — the other single modifier keys are all spoken for by the system.

See also: Accidental Tech Podcast.

Wednesday, January 8, 2020 [Tweets] [Favorites]

LaunchCuts

Federico Viticci:

Like the aforementioned Shortcuts utilities, LaunchCuts was born of its developer’s frustration with the lack of folders in Shortcuts – a basic feature that is still bafflingly absent from the app in 2020.

[…]

LaunchCuts can only read data from your local Shortcuts library by running an additional helper shortcut inside the Shortcuts app. This shortcut is based on an advanced technique that uses the native, Apple-developed ‘Get My Shortcuts’ action to generate a list of all your installed shortcuts and extract metadata from each. I’m oversimplifying what the LaunchCuts Helper does, but, essentially, the shortcut scans the entire contents of the Shortcuts app and parses details such as the names of your shortcuts, their colors and glyphs, whether you’re using them as widgets or action extensions in the share sheet, and even the contents of certain actions contained inside them (more on this later).

[…]

There are caveats to this approach. In addition to being dependent on a technique that Apple may easily close off in the future (see footnote above), it means you’ll need to run the LaunchCuts Helper shortcut manually and periodically to give the LaunchCuts app a fresh database with your latest shortcuts and modifications to existing ones.

Twitter Will Add Options to Limit Replies

Dieter Bohn (via MacRumors):

Xie says Twitter is adding a new setting for “conversation participants” right on the compose screen. It has four options: “Global, Group, Panel, and Statement.” Global lets anybody reply, Group is for people you follow and mention, Panel is people you specifically mention in the tweet, and Statement simply allows you to post a tweet and receive no replies.

“Getting ratio’d, getting dunked on, the dynamics that happen that we think aren’t as healthy are definitely part of ... our thinking about this,” Xie says. When asked if there’s a concern if the ability to limit replies could mean misinformation couldn’t be as easily rebutted, Xie gestured to the ability to quote tweet as one possible resolution, but it’s “something we’re going to be watching really closely as we experiment.”

I don’t see how quote tweets would really help with that problem, since people would have no way of finding them.

Previously:

Update (2020-01-10): Mike Rockwell:

I mean, this does seem incredibly easy to bypass. Presumably, you could simply mention the person who published the Statement and not give your tweet the reply distinction. You could also just add a link to the statement to specifically reference what you’re “replying” to.

Benjamin Mayo:

Brands are definitely going to be making use out of the Statements mode. Below almost every brand tweet I see, often when the brand has paid for the tweet to be promoted to a wider audience, are replies from people complaining about something about the company’s products that is completely unrelated to the tweet content.

[…]

A Statement option would close that hole and make promoted posts much more like traditional display advertising. A public placard with no interaction.

Separately, I think Twitter certainly risks losing some of its ‘community’ if all celebrities suddenly switch to posting in Statement mode and thereby hiding all reactions to their tweets. I find a lot of the fun of Twitter is that feeling of everyone being able to jump in the same conversation.

Chosen-Prefix Collision for SHA-1

SHA-1 is a Shambles (via Hacker News):

We have computed the very first chosen-prefix collision for SHA-1. In a nutshell, this means a complete and practical break of the SHA-1 hash function, with dangerous practical implications if you are still using this hash function. To put it in another way: all attacks that are practical on MD5 are now also practical on SHA-1. Check our paper here for more details.

See also: Bruce Schneier.

Previously:

Update (2020-01-10): Git has been working on the SHA-1 problem since 2017, but it seems like the default behavior is still to use it. Here’s some information on the efforts. A recent post on the Git mailing list about the chosen-prefix collision did not generate much interest or a definitive statement.

Xcode Preview Snips

Jordan Morgan:

Look, you know where I’m going with this. If you’ve adopted SwiftUI (or even if you haven’t - view controllers apply here too) then you know Xcode Previews are more than a time saver. They are a fork in the road. There’s no going back once you get hooked on that instant feedback.

Today, I’ll share a few quick snips of my go-to previews. Some of these are already well known, tweeted and blogged about - but my topic for this post is my favorite things to use with PreviewProvider, so I’ve included them anyways for posterity’s sake. Let’s take a look.

Tuesday, January 7, 2020 [Tweets] [Favorites]

Ten Years of Apple on One Page

Benjamin Mayo:

Apple entered the 2010s just as the iPhone began to explode in popularity. The iPhone became the most successful consumer product, ever. Sales surged for another five years and still make up a majority of Apple’s revenues.

[…]

In an on-stage interview a couple months after the iPad was released, Jobs told Walt Mossberg and Kara Swisher his vision of the future of the computer industry, comparing PCs to ‘specialist’ trucks and iPads to mainstream cars.

[…]

The first-generation MacBook Air was sleek but slow and expensive. Apple’s second stab at the ultrabook was a huge success.

[…]

iCloud is now a foundational feature across all of Apple’s platforms but it didn’t start to become a thing until 2011 with iOS 5.

[…]

Apple brought Retina to the iPhone in 2010 but it took another two years for the high-density screens to make their way to the iPad and the Mac.

[…]

The culmination of Jobs demise, Forstall’s ousting and Ive’s new found mandate over all of Apple human interface resulted in iOS 7. Apple redesigned the entire aesthetic of its mobile operating system in about eight months.

[…]

Swift debuted with big ambitions to be a universal programming language spanning app development to low-level systems programming, with expressive syntax and more safety guarantees than Objective-C could provide, whilst also pushing for bare metal performance and efficiency goals.

[…]

The 15-inch MacBook Pro was the straw that broke the camel’s back, as far as Apple’s alignment with the professional market. It would begin to rectify the relationship and appeal of its pro products in 2017 but a ‘truly great’ MacBook Pro would not be available for another three years.

[…]

As a financial event, this would just be an amusing statistic of history but of course this result had ramifications on Apple’s product plans. Apple doubled down on Services initiatives as a way to show investors underlying long-term growth potential in its business.

OpenDrop

OpenDrop (via André Staltz):

OpenDrop is a command-line tool that allows sharing files between devices directly over Wi-Fi. Its unique feature is that it is protocol-compatible with Apple AirDrop which allows to share files with Apple devices running iOS and macOS. Currently (and probably also for the foreseeable future), OpenDrop only supports sending to Apple devices that are discoverable by everybody as the default contacts only mode requires Apple-signed certificates.

[…]

OpenDrop is experimental software and is the result of reverse engineering efforts by the Open Wireless Link project.

Safari’s “Reload Page From Origin”

Jeff Johnson (tweet):

It turns out that “Reload Page” does not actually reload the page in the way you expect. I’m not sure exactly what “Reload Page” does, but it still seems to rely on the disk cache. If you hold down the option key, you see “Reload Page” replaced in the menu by “Reload Page From Origin”, which is the reload you expect, the one that ignores the disk cache and loads everything again from the web.

I’m not sure how to do this in iOS. I thought may be a long-press would give me options, but it doesn’t.

I wish that Safari would take a cue from Firefox and Google Chrome in allowing fine-grained control over cookies. Safari has per-site preferences for Auto-Play, Downloads, Notifications, etc., but it doesn’t have per-site preferences for cookies. Compare with Firefox and Google Chrome shown below. The best feature they have is to clear cookies when you quit the app, a feature I wish that Safari would adopt too.

See also: Melissa Holt.

Monday, January 6, 2020 [Tweets] [Favorites]

Mystery Scrollwheel Crash

Brent Simmons:

For NetNewsWire for Mac, I get one or two crash logs a week referencing scrollView:​scrollWheelWithEvent:.

Here’s the bug for it.

I’ve been getting these crashes in my app, too, but only on macOS 10.14. On previous macOS versions, the app is not using responsive scrolling. And I’ve yet to see this crash from anyone using macOS 10.15. I’ve never seen it on my Macs.

Beware Spinlocks in User Space

Malte Skarupke (via Shantonu Sen, Niels Broekhuijsen):

I overheard somebody at work complaining about mysterious stalls while porting Rage 2 to Stadia. […] The only thing those mysterious stalls had in common was that they were all using spinlocks. I was curious about that because I happened to be the person who wrote the spinlock we were using. The problem was that there was a thread that spent several milliseconds trying to acquire a spinlock at a time when no other thread was holding the spinlock. Let me repeat that: The spinlock was free to take yet a thread took multiple milliseconds to acquire it. […] In our case we were able to make the problem go away by replacing spinlocks with mutexes, but that leads to the question: How do you even measure whether a spinlock is better than a mutex, and what makes a good spinlock?

Linus Torvalds (Hacker News):

So now you still hold the lock, but you got scheduled away from the CPU, because you had used up your time slice. The “current time” you read is basically now stale, and has nothing to do with the (future) time when you are actually going to release the lock.

Somebody else comes in and wants that “spinlock”, and that somebody will now spin for a long while, since nobody is releasing it - it’s still held by that other thread entirely that was just scheduled out. At some point, the scheduler says “ok, now you’ve used your time slice”, and schedules the original thread, and now the lock is actually released. Then another thread comes in, gets the lock again, and then it looks at the time and says “oh, a long time passed without the lock being held at all”.

[…]

You’re just getting random values because different schedulers have different heuristics for “do I want to let CPU bound processes use long time slices or not”?

[…]

Notice, how when the author uses an actual std::mutex, things just work fairly well, and regardless of scheduler. Because now you’re doing what you’re supposed to do. Yeah, the timing values might still be off - bad luck is bad luck - but at least now the scheduler is aware that you’re “spinning” on a lock.

Malte Skarupke:

Once we break it down like that we realize that actually these are all the same case. In all of these cases one thread can run, all others are calling yield(). The only difference between the case that I wanted to measure and the other two accidental cases is whether the scheduler is incorrectly not running thread C or incorrectly not running thread N. In either case all other fifteen threads are just calling yield().

So your claim is that it’s a problem that I try to measure how long it takes for thread N to run even though I might accidentally be measuring how long it takes for thread C to run. But I claim that that’s fine because these are all equally problematic. One thread could run, all other threads are yielding, yet that one thread is not running. And we don’t care whether the thread that could run is thread N or thread C.

Linus Torvalds:

The problem with that is “yield” is pretty much undefined. The definition of it is literally about single queue of a real-time behavior with a real-time scheduler with priorities.

[…]

What you want to use it for is “schedule the right process”. But you don’t even know what the right process is, or if you do you don’t tell the system (because sched_yield() literally doesn’t have that interface), so the kernel has to guess.

Previously:

Copyright © 2000–2020 Michael Tsai.