Archive for November 6, 2018

Tuesday, November 6, 2018

String’s ABI and UTF-8

Michael Ilseman:

We just landed String’s proposed final ABI on master. This ABI includes some significant changes, the primary one being that native Swift strings are stored as UTF-8 where they were previously stored either as ASCII or UTF-16 depending on their contents.


Unifying the storage representation for ASCII and Unicode-rich strings gives us a lot of performance wins. These wins are an effect of several compounding factors including a simpler model with less branching, on-creation encoding validation of native Strings (enabled by a faster validator), a unified implementation code path, a more efficient allocation and use of various bits in the struct, etc.


By maintaining nul-termination in our storage, interoperability with C is basically free: we just use our pointer.


Efficient interoperability with Cocoa is a huge selling point for Swift, and strings are lazily bridged to Objective-C. String’s storage class is a subclass of NSString at runtime, and thus has to answer APIs assuming constant-time access to UTF-16 code units. We solved this with a breadcrumbing strategy: upon first request from one of these APIs on large strings, we perform a fast scan of the contents to check the UTF-16 length, leaving behind breadcrumbs at regular intervals. This allows us to provide amortized constant-time access to transcoded UTF-16 contents by scanning between breadcrumbs.


The branch also introduces support in the ABI (but currently not exposed in any APIs) for shared strings, which provide contiguous UTF-8 code units through some externally-managed storage. These enable future APIs allowing developers to create a String with shared storage from a [UInt8], Data, ByteBuffer, or Substring without actually copying the contents.

This sounds great. Here is the pull request and the implementation of StringObject (which is actually a struct). Note that this locks in the in-memory layout for Swift strings, because code that uses them may be inlined. In contrast, Cocoa’s NSString has an internal representation that has evolved over time. This was possible because access was indirected through the Objective-C runtime.

Previously: Swift String ABI, Performance, and Ergonomics, Swift ABI Stability Dashboard.

Update (2018-11-07): See this pull request from David Smith about improving performance for bridged strings (tweet).

Update (2018-11-09): See also: Hacker News.

Update (2018-12-11): Michael Ilseman:

Would you like to access the raw UTF-8 code units backing a String? Now you can, thanks to String.UTF8View.withContiguousStorageIfAvailable hook[…]

Flickr to Limit Free Accounts to 1,000 Photos

Andrew Stadlen:

Beginning January 8, 2019, Free accounts will be limited to 1,000 photos and videos. If you need unlimited storage, you’ll need to upgrade to Flickr Pro.

In 2013, Yahoo lost sight of what makes Flickr truly special and responded to a changing landscape in online photo sharing by giving every Flickr user a staggering terabyte of free storage. This, and numerous related changes to the Flickr product during that time, had strongly negative consequences.

First, and most crucially, the free terabyte largely attracted members who were drawn by the free storage, not by engagement with other lovers of photography.


Giving away vast amounts of storage creates data that can be sold to advertisers, with the inevitable result being that advertisers’ interests are prioritized over yours. Reducing the free storage offering ensures that we run Flickr on subscriptions, which guarantees that our focus is always on how to make your experience better.

Don MacAskill:

Actually, more than 97% of @Flickr Free accounts are under 1,000 photos. The changes we announced today are mostly about enhancing Flickr Pro. Less than 3% of Flickr Free accounts, who chew up a huge amount of storage & costs that others must bear, are asked to make a decision.

This makes sense except that:

A. Lee Bennett Jr.:

All the people complaining of the @Flickr changes and are leaving are probably the ones that will make Flickr a better community when they leave.

Hope y’all enjoy Google Photos’ compression and watching your original/uncompressed photos disappear.

Previously: SmugMug Acquires Flickr.

Update (2018-11-08): Don MacAskill:

The Flickr Commons photos (those uploaded by the archival, governmental, etc. institutions we are working with) are safe. We are extremely proud of these partnerships. These photos won’t be deleted as a result of any of our announced changes. The only reason they’d disappear is if the organization that uploaded them decided to delete them.

Photos that were Creative Commons licensed before our announcement are also safe. We won’t be deleting anything that was uploaded with a CC license before November 1, 2018. Even if you had more than 1,000 photos or videos with a CC license.

Update (2018-11-13): Glenn Fleishman:

Do you know someone who died and their Flickr account remains at free tier, and they have > 1,000 photos and videos? I’d suggest contacting their family, and then contacting Flickr. In an interview with SmugMug head Don MacAskill he said several people raised the notion of accounts of deceased people losing images, but on asking, he as of a few days ago hadn’t been given any actual account names.

Nick Heer:

Pretty wild that this Flickr email announcing that they’ll be deleting some of my photos in three months buries that rather critical detail in small grey text near the bottom.

Update (2019-01-14): Brian Matiash:

If you think about it, 2018 has been a very turbulent year for photographers and the social media platforms we frequent. We’ve poured one out for Google+, Facebook has had its trust shattered after a series of PR and political nightmares, 500px was sold to the “Getty of China,” and Instagram has all but nuked originality and organic growth. So, you could imagine the vacuum that is being created for photographers who are looking for a proven place to share their work in ultra-high resolution (Flickr will soon support photo resolutions up to 5,120 x 5,120) and use embedded color profiles (that’s coming soon, too), while also being a part of authentic communities.


I, for one, am very excited to see new life being breathed into Flickr and strongly believe that SmugMug is the right company to do so.

Don MacAskill:

Here’s your fun fact of the day: Every major tech company you can think of (Google, Apple, Amazon, Adobe, Microsoft, …) have wanted to acquire us. I’ve never even heard the price. They haven’t answered my first Q properly: “Who are our customers and how will you care for them?”

David Heinemeier Hansson:

The best part of Flickr’s new paid model is that going pro means that your FOLLOWERS get to avoid being showered with ads. That’s probably what felt most dirty about feeding Instagram. You’re giving Facebook the bait to hook others with. Ugh.

Brian Matiash:

While you may cry and lament that the ad-free @Flickr experience requires a $50/yr Pro sub, at least you have that option. You think FB/IG would ever let you pay $50/yr to suppress those incessant ads? Absolutely not. Options are good. Consumers win w/options.

Nicole S. Young:

However, something has changed, I can feel it. It’s similar to that anticipatory sensation you get when you’re standing outside and the air shifts just before a storm. Photographers are talking about Flickr again! People are sharing photos, and replying to discussions in groups! It still has a way to go before it is really back to what it was (and hopefully even bigger and better than its glory days), but it is going to get there.


I’m hopeful that groups become a really big part of Flickr again. But right now it’s extremely challenging to find a group that is active. Many of them may have photos recently added to the group’s photo pool, but if you look at the discussion many of them have not been updated in many, many years.

Dave Winer:

I requested to download all the info Flickr has about me on Tuesday, got the data on Wednesday, and yesterday I spent a couple of hours trying to figure out what I got.

Update (2019-01-23): Chuq Von Rospach:

I’d like to see them take this a step further as part of the work to revitalize groups: remove the ability to post photos to a group anywhere but the group page itself. Right now, you can do it from an image page, and I think that encourages mindless image dumping, because you don’t actually have to interact the group to dump images on it. That just encourages spamming of images, which we want to see go away. So make people click into a group to post an image, which will both discourage wide spamming (or at least make them waste more time doing it) and get them on the group page to see the group rules and the discussion areas.

Discussion areas are still in most groups a wasteland and a ghost town, and I want to see Smugmug do some updating here to make them more interesting and relevant. They seem to be making good progress on the general forum spam problem, which is nice.

I’d also suggest they identify all of the groups where the admins are no longer active on flickr, or groups who’s admins have left flickr — and delete them. Flickr groups would be a stronger feature with 50% fewer groups seeking attention, honestly. Or maybe 90% fewer. Clear out all of the deadwood, and let the new Flickr population start building a new community there.

Anil Dash:

It’s been largely overlooked that FOMO was coined by Caterina Fake, a cofounder of Flickr — one of the very first people who ever helped create a large-scale social network for photo sharing. Her comments on FOMO came less than 6 months after Instagram launched. Though of course both services seemed superficially similar because they were social networks built around photos, Instagram’s social design was almost always with the opposite intent of Flickr’s social goals. It was almost as if Instagram was designed to optimize for FOMO.

Apple’s New Map

Justin O’Beirne:

If RMSI is creating Apple’s buildings by manually tracing them from satellite imagery, it would explain how Apple’s building perimeters could be more precise than Google’s algorithmically-generated buildings:


Regardless of how Apple is creating all of its buildings and other shapes, Apple is filling its map with so many of them that Google now looks empty in comparison[…] And all of these details create the impression that Apple hasn’t just closed the gap with Google—but has, in many ways, exceeded it...

...but only within the 3.1% of the U.S. where the new map is currently live.


In other words, TomTom’s database somehow has roads from Parkfield’s boomtown days—roads that have been gone for more than 75 years. No wonder why Apple removed them.


Unless they’re already listed on Yelp, none of the shapes Apple has added appear in its search results or are labeled on its map. And this is a problem for Apple because AR is all about labels—but Apple’s new map is all about shapes.

So is Apple making the right map?

James O’Leary:

Apple: “We don’t think there’s anybody doing this level of work that we’re doing.”

TechCrunch: “wow lemme write that”

Ex-Apple Maps, politely: you offshored maps to 5,000 doing manual work. after 5yrs, you have vegetation for 3.1% of the US +bad place data

John Gruber:

O’Beirne’s keen observation is this: even in the areas where Apple’s new maps have rolled out, Google is still far ahead in correctly identifying places and specific destinations.

Nick Heer:

Rebuilding Maps in such a comprehensive way is going to take some time, so I read O’Beirne’s analysis as a progress report. But, even keeping that in mind, it’s a little disappointing that what has seemingly been prioritized so far in this Maps update is to add more detailed shapes for terrain and foliage, rather than fixing what places are mapped and where they’re located. It

Rui Carmo:

At this rate, we are never going to be able to use Apple Maps in Europe.

Although Apple Maps have improved markedly (and worked OK during my recent trip to Edinburgh) there are still loads of things missing (landmarks, streets, transit info), and have stayed that way for years, whereas Google Maps are sometimes corrected within days.

Previously: Rebuilding Apple Maps Using Apple’s Own Data, Google Maps’s Moat.

Update (2018-11-09): See also: Hacker News.