Archive for August 18, 2021

Wednesday, August 18, 2021

NeuralHash Implementation and Collision

Joseph Cox et al. (Slashdot, Hacker News, Reddit):

On Wednesday, GitHub user AsuharietYgvar published details of what they claim is an implementation of NeuralHash, a hashing technology in the anti-CSAM system announced by Apple at the beginning of August. Hours later, someone else claimed to have been able to create a collision, meaning he tricked the system into giving two different images the same hash.

Juli Clover:

In a statement to Motherboard, Apple said that the version of the NeuralHash that Yvgar reverse-engineered is not the same as the final implementation that will be used with the CSAM system.


Matthew Green, who teaches cryptography at Johns Hopkins University and who has been a vocal critic of Apple’s CSAM system, told Motherboard that if collisions “exist for this function,” then he expects “they’ll exist in the system Apple eventually activates.”

“Of course, it’s possible that they will re-spin the hash function before they deploy,” he said. “But as a proof of concept, this is definitely valid,” he said of the information shared on GitHub.

Hector Martin:

“Early tests show that it can tolerate image resizing and compression, but not cropping or rotations.”

Like every other perceptual image hash. It’ll also have collisions. Keep in mind that the matching is fuzzy (you have to allow some wrong bits).

It’s not hard at all to attack such a hash to make it produce false positives.

Say I am law enforcement and I want access to your photos. I send you >30 messages with non-CSAM but colliding images. Your phone now thinks you have CSAM and grants Apple access to your data.

Then I just have to subpoena Apple for the data they already have, and I have your photos.

Meanwhile the people who actually have CSAM just have to add a frame to their images to completely neuter the system.

A lot rests on how much we can trust Apple’s human reviewers.

Also, apparently Apple’s neural network, by virtue of having 200+ (!) layers and due to floating point rounding issues, actually produces wildly different hashes on different hardware (9 bits difference between iPad and M1 Mac!). That’s... garbage. That’s 9 bits of match noise.


Actually, how does this even work at all? You have to do fuzzy matching of perceptual image hashes like NeuralHash. But they’re doing some PSI crypto stuff after that that would seem to be incompatible with it, and at no point do they talk about this.

This is not a thing. This cannot mathematically be a thing. There is no way to design a perceptual image hash to always result in the same hash when the image is altered in small ways. This is trivial to prove.

Bruce Schneier:

This was a bad idea from the start, and Apple never seemed to consider the adversarial context of the system as a whole, and not just the cryptography.

Russell Brandom:

In a call with reporters regarding the new findings, Apple said its CSAM-scanning system had been built with collisions in mind, given the known limitations of perceptual hashing algorithms. In particular, the company emphasized a secondary server-side hashing algorithm, separate from NeuralHash, the specifics of which are not public. If an image that produced a NeuralHash collision were flagged by the system, it would be checked against the secondary system and identified as an error before reaching human moderators.


But actually generating that alert would require access to the NCMEC hash database, generating more than 30 colliding images, and then smuggling all of them onto the target’s phone.


Update (2021-08-21): See also: Hacker News.

Bruce Schneier:

I’m not convinced that this secondary system was originally part of the design, since it wasn’t discussed in the original specification.

Sarah Jamie Lewis:

The Apple system dedupes photos, but burst shots are semantically different photos with the same subject - and an unlucky match on a burst shot could lead to multiple match events on the back end if the system isn’t implemented to defend against that.

Jonathan Mayer:

We wrote the only peer-reviewed publication on how to build a system like Apple’s — and we concluded the technology was dangerous. We’re not concerned because we misunderstand how Apple’s system works. The problem is, we understand exactly how it works.

Brad Dwyer (via Hacker News):

In order to test things, I decided to search the publicly available ImageNet dataset for collisions between semantically different images.


There were 2 examples of actual collisions between semantically different images in the ImageNet dataset.

Update (2021-09-08): (via Hacker News):

Given that it’s possible to generate a false positive, it is also possible to deliberately create images that match a given hash. So, for example, someone who wants to get another person in trouble can send them innocent-looking images (like images of kittens) and manipulate those images to match a hash of known CSAM.

This site is a proof of concept for collision attacks. The images of the kittens are manipulated to match the hash of the image of the dog (59a34eabe31910abfb06f308). As a result, all images shown on this page share the same hash. When these images are both hashed with the Apple NeuralHash algorithm, they return the same hash.

Touché 1.1.5

Daniel Jalkut:

Over the years the system’s support for Touch Bar changed in ways that made Touché slowly become less reliable and, finally, to more or less not work on any Macs at all.

I recently had some insights about how the Touch Bar support on the system has changed, and was able to put together an update to Touché that restores functionality.


Safari 15 Changes in Beta 6

Juli Clover (tweet):

Throughout the beta testing period, Apple has been tweaking the design of the Safari browser on the iPhone and in beta 6, there are further refinements. The bottom tab bar has been redesigned to appear below page content, and Apple has also added a toggle to show the address bar at the top of the iPhone rather than the bottom.


With the bottom view option toggled on, Safari offers a dedicated toolbar with buttons at the bottom of the interface, which is also an improvement over the prior floating design.

Apple has also introduced new setting options to remove the website tinting and to enable a Tab Bar while in landscape mode. There was previously a “Show Color in Tab Bar” accessibility setting, which appears to be the same as the new “Allow Website Tinting” toggle.

Federico Viticci:

It takes 5 seconds to see that this new Safari design is much better. Instantly clicked with me.

Bringing back a toolbar allows easier access to controls. More views are using the half-sheet style. Putting the URL bar at the top reverts to the old Safari. All of this is great.

Joe Cieplinski:

Wow. That…looks like crap.

Either do a new feature or don’t. I feel like this option to leave everything the way it was is more confusing.

Ryan Jones:

I’ve been trying to tell y’all that fast tab switching is clearly a design requirement for iOS 15 Safari!

I like this design – looks at home and works well.

Matthew Panzarino:

Very strong “fine, whatever” vibes from the new ‘below’ mode. And the new ‘option’ is basically “ i’m sorry I’m sorry I’m trying to remove it”

I think the way to read this is ‘intermediate step to where we want to go eventually and we’ll see you in iOS 16’. My bet is that the ‘url bar at top’ option stays for years though.

Hopefully the one lesson learned that sticks though is not to bury high traffic actions under additional layers for very little win aside from aesthetics.

Nick Heer:

These are incremental changes to a big redesign, and I think they create the most successful iteration yet. Bringing the toolbar to the bottom is undeniably a muscle memory breaker, but I think it is worth the cost because it keeps a user’s hands in the same position more often. You can go from scrolling through a webpage to entering a URL without once shuffling the device in your hands.

This new version still has some rough edges. The huge drop shadow around the address bar is a nonstandard effect that confuses me. I guess it is supposed to indicate that the element is floating and interactive, but it creates a kind of blurry grey mess. The drop shadow also visually disconnects the address bar from the page or tab it represents.

Curtis Herbert:

You can tell this is the mostly final Safari design for iOS because the animations are all polished AF.

Juli Clover:

Safari Technology Preview release 130 includes bug fixes and performance improvements for Web Inspector, CSS, JavaScript, Media, Web API, and IndexedDB.


Update (2021-08-21): John Gruber:

In a very real sense, the system worked. It’s good that Apple tried something ambitious and original with the layout for Safari on iPhone. The reason for the trend toward moving more navigation controls to the bottom of the screen is obvious: our phones are bigger than ever (iPhone 12 Mini aside), and our hands aren’t growing. It’s also good that Apple was receptive to the feedback from those using the developer and public betas. They listened, they fixed the design to address the problems, and here we are, with a layout for Mobile Safari that I think is better than ever. (I hedge with “I think” only because it just shipped — my opinions aren’t fully formed.)

The unusual part is that we got to see Apple’s design process play out in public.

Update (2021-09-08): Jason Snell:

The design of Safari 15 on the iPhone has gone to a better place, but Stephen Hackett reminds us that trouble on the Mac and iPad remain[…]