Friday, July 29, 2016 [Tweets]

Developer Tools Survey

Dave DeLong:

Survey: what non-apple dev tools are you still using, 6+ months after you first got them?

I’m currently using: BBEdit, FogBugz (in Google Chrome), Git and Tower, LaunchBar, Dash, make, Python, VMware Fusion, Hopper, Kaleidoscope, Hex Fiend, xScope, Deploymate, RB App Checker Lite, Script Debugger, SourceTree, Base, Paw, Soulver, and OmniFocus. I plan to switch from Xcode bots to a third-party continuous integration solution.

For the non-code parts of my apps, I also use reStructuredText, pdfLaTeX, Counterparts Lite, Acorn, Icon Slate, and Opacity.

And for deployment: DropDMG and rsync.

Python Internals: PyObject

Sergei Danielian (via Hacker News):

Each Python’s type implementation (PyIntObject, PyFloatObject or PyDictObject) has PyObject_HEAD located as its first member (or the first member of its first member, and so on). This member sub-object is guaranteed to be located at the same address as the full object.

The PyObject_HEAD refers at that member sub-object, but could be cast to the full type once ob_type has been inspected to get knowledge of what the full type is.

Philip Guo:

Here are nine lectures walking through the internals of CPython, the canonical Python interpreter implemented in C. They were from a dynamic programming languages course that I taught in Fall 2014 at the University of Rochester. The format isn’t ideal, but I haven’t seen this level of detail about CPython presented online, so I wanted to share these videos.

Jake Vanderplas:

We saw above the extra type info layer when moving from a C integer to a Python integer. Now imagine you have many such integers and want to do some sort of batch operation on them. In Python you might use the standard List object, while in C you would likely use some sort of buffer-based array.

A NumPy array in its simplest form is a Python object build around a C array. That is, it has a pointer to a contiguous data buffer of values. A Python list, on the other hand, has a pointer to a contiguous buffer of pointers, each of which points to a Python object which in turn has references to its data (in this case, integers).

Don’t Trust Sourceforge Downloads


In addition to injecting malware into their downloads (a practice they claim, hopefully truthfully, to have stopped), Sourceforge also presents an initial download page over HTTPS, then redirects the user to HTTP for the download itself, snatching defeat from the jaws of victory. This is fantastically irresponsible, especially for a site offering un-sandboxed binaries for download, especially in the era of Let’s Encrypt where getting a TLS certificate takes approximately thirty seconds and exactly zero dollars.

Previously: What Happened to SourceForge?

SwiftKey Keyboard Leaked User Information to Strangers

Cara McGoogan (via MacRumors, Hacker News):

A British keyboard app that uses artificial intelligence to predict the next word you want to write has suspended part of its service after users reported receiving predictions meant for other people, including email addresses and phone numbers.


“A few days ago, I received an email from a complete stranger asking if I had recently purchased and returned a particular model of mobile phone, adding that not one but two of my email addresses (one personal and one work address) were saved on the phone she had just bought as brand-new,” said the user.

The stranger with the new phone went through each letter of the alphabet and told the user the suggested words. They included names of the user’s friends and addresses for private servers they connect to for work.

Previously: iOS 8 Keyboards.

Update (2016-07-29): Rosyna Keller:

But congratulations to Microsoft’s SwiftKey for proving exactly why you should heed the full access warning!

Thursday, July 28, 2016 [Tweets]

Apple User Interface Trends

Nick Babich:

For every action, there is a reaction. And in the world of digital design, flat design was a reaction against skeumorphism. Since anything on a screen will never truly look three dimensional, why not to stripping away illusory decoration and focus on functionality instead?


Recently, designers have begun to realize the usability issues of flat design and as a result, a more balanced interpretation of flat design has emerged. New solution sometimes referred to as ‘almost flat’ or ‘flat 2.0’ design. This design style is mostly flat, but makes use of subtle shadows, highlights, and layers to create some depth in the UI.

Nicholas W. Howard (via Rich Siegel and Gus Mueller):

In Apple’s view, an icon depicting a camera and a photo was too literal for an application that handles photos. Therefore, when iPhoto gave way to its replacement in 2015—Photos—the previous carefully-rendered icon gave way to this bland, meaningless rainbow abstraction.


Contrary to the ideas they express on this archival Mac Basics page (which, it must be observed, still features old icons), Apple nowadays prefers abstraction to metaphor, the less literal to the more literal, the two-dimensional glyph to the three-dimensional illustration. Allow me to explain how this ideological shift occurred in full-step with the design community at large.


Apple has gradually drained the color from the Finder sidebar, removing helpful visual distinctions from a part of OS X that millions of people interact with every day.

Nicholas W. Howard:

Now turn your attention to the barber pole’s successor, on the right. This redesigned loading signal, introduced in OS X Yosemite, reduces visibility in several significant ways: it measures less than half the height of its predecessor, uses only blue rather than blue-and-white, and makes its animated element—a slight glowing patch that moves across it—so bafflingly faint that even someone with superb vision can barely see it. The animation is the part that conveys “loading”: if Apple fails to make the “loading” state obvious, how will anyone ever know the loading is happening at all?


Clicking this “New Folder” button (if one can accurately call it a button), found in the Notes app, yields insignificant feedback: it almost feels as if nothing happens.


Hovering over Safari’s address bar no longer turns the cursor from the regular pointer to the “I-beam” text cursor; of course, one can still click on the address bar and type, but the cursor swap that whispered, “This is a place for typing,” is gone. In my testing, various search fields throughout the system sometimes replicated this problem, and sometimes not. OS X once handled this behavior properly and consistently. It no longer does.


It is a pity you did not know the “Detect Displays” button still exists. The button is invisible. You need only… hold down the Option key to summon it.


Some of these buttons in Pages have an arrow to indicate a dropdown menu. Others have no arrow but spawn a dropdown menu anyway. Apple has presented us with a guessing game.

John Gruber:

Luke Wroblewski posted an interesting side-by-side comparison of the Today view, Control Center, and standard sharing sheets in iOS 7 and the iOS 10 public beta. Much less transparency, more solid shapes in place of outlines, and more use of color.

End of Source-breaking Changes for Swift 3

Ted Kremenek:

The Swift team at Apple has reflected on this and decided what it “means” for Swift 3 to be source compatible with Swift 4 and later releases going forward. Our goal is to allow app developers to combine a mix of Swift modules (e.g., SwiftPM packages), where each module is known to compile with a specific version of the language (module A works with Swift 3, module B works with Swift 3.1, etc.), then combine those modules into a single binary. The key feature is that a module can be migrated from Swift 3 to 3.1 to 4 (and beyond) independently of its dependencies.

While the exact details of how we will accomplish this feat are still being discussed, here is a sketch of how this will likely work in the Swift 4 timeframe. The key enabler is a new compiler flag that indicates the language version to compile for (e.g., similar to the clang -std=c99 flag). The compiler flag will be provided by the build system you are using (e.g., Xcode, SwiftPM, etc.) on a per-module basis[…]


The great thing about this approach is that a single Swift 4 compiler is building all of the sources in an application. This allows us to roll out this approach before achieving full ABI stability — something that will be a goal for Swift 4, but is impractical to achieve for a Swift 3.x release. It also provides us a general framework in the future for handling source compatibility as Swift evolves.

See also: Swift Programming Language Evolution: Proposal Status (via Jacob Bandes-Storch).

Witch, Spaces, and Private API

Rob Griffiths:

Since that rejection, we’ve been looking for a solution to providing Spaces support for App Store Witch users … and unfortunately, we couldn’t find one: The only way we can get the window information we need from other Spaces is to use the private API. And that means that only the direct version of Witch supports Spaces, because we can use the private API. But App Store Witch no longer supports Spaces.


Gwendal Roué:

GRDB.swift is a Swift application toolkit that provides access to SQLite databases.


GRDB is protocol-oriented, and fetching methods are proper to the RowConvertible protocol.


This makes GRDB quite unlike other Swift ORMs that use class inheritance and make heavy use of objects mutability, like Realm and Core Data. Both of them provide uniquing and auto-updating records.

Wednesday, July 27, 2016 [Tweets]

LastPass URL Parsing Bug

Mathias Karlsson (Hacker News, Slashdot):

Stealing all your passwords by just visiting a webpage. Sounds too bad to be true? That’s what I thought too before I decided to check out the security of the LastPass browser extension.


I reported this to LastPass through their responsible disclosure page and the report was handled very professionally. The fix was pushed in less than a day(!), and they even awarded me with a bug bounty of $1,000.


Should we stop using password managers? No. They are still much better than the alternative (password reuse).

Although, taking a second to disable autofill functionality is a good move because this isn’t the first autofill bug we’ve seen, and I doubt it will be the last.


Disclosure: I work for AgileBits, makers of 1Password.

For browser extensions, the URL constructor would be even easier [for parsing]. (Yes, I know it says that IE doesn’t support it, but IE doesn’t have a proper extensions framework, so it’s irrelevant to this topic.)

Adobe Direct Download Links


The difference is that direct links to download the complete standalone/offline installers are not possible and no longer work if (only) the new approach is used. We at ProDesignTools have a long history of providing direct download links to all major Adobe software products, but now there is no way that we (or anybody) can make direct download links available for the new CC products released today (and beyond)!

[UPDATE (June 22nd @ 7pm) – We were just contacted by an Adobe Product Manager who says the company is aware of the situation and hoping to find a solution to restore direct download links and standalone installers for all tools in the new release! So we’re happy to report they are listening to customers and hearing your feedback. Please stay tuned to this page where we will keep you updated in the coming days!]

I had been using this page to get direct download links for Lightroom updates, but it hasn’t been updated in a while. I eventually found in a comment that the Lightroom 6.6.1/CC 2015.6.1 update is available here.

EFF DMCA Lawsuit

Matthew Green:

Today I filed a lawsuit against the U.S. government, to strike down Section 1201 of the Digital Millennium Copyright Act. This law violates my First Amendment right to gather information and speak about an urgent matter of public concern: computer security. I am asking a federal judge to strike down key parts of this law so they cannot be enforced against me or anyone else.


There’s a saying that no good deed goes unpunished. The person who said this should have been a security researcher. Instead of welcoming vulnerability reports, companies routinely threaten good-faith security researchers with civil action, or even criminal prosecution. Companies use the courts to silence researchers who have embarrassing things to say about their products, or who uncover too many of those products’ internal details. These attempts are all too often successful, in part because very few security researchers can afford a prolonged legal battle with well-funded corporate legal team.


In the United States, one of the most significant laws that blocks security researchers is Section 1201 of the Digital Millennium Copyright Act (DMCA). This 1998 copyright law instituted a raft of restrictions aimed at preventing the “circumvention of copyright protection systems.” Section 1201 provides both criminal and civil penalties for people who bypass technological measures protecting a copyrighted work. While that description might bring to mind the copy protection systems that protect a DVD or an iTunes song, the law has also been applied to prevent users from reverse-engineering software to figure out how it works. Such reverse-engineering is a necessary party of effective security research.

Removing Bit Flags in Swift Option Sets

Erica Sadun:

This code creates the complete .forbidAll set and then removes the local restriction.

var restrictions: AVAssetReferenceRestrictions = [ .forbidAll ]
restrictions .remove(.forbidLocalReferenceToLocal)

Interestingly, you can also pass .forbidAll without brackets in the current version of Swift and it will compile. […] I’m told that this option set syntax works because each element of an option set is itself an option set: [.forbidAll] is the same type and equal to .forbidAll. The array literal form of [.a, .b, .c] is syntactic niceness for creating an empty option set and then inserting (i.e. bitwise OR) each element)

Or you can write:

let restrictions: AVAssetReferenceRestrictions = .forbidAll.subtracting(.forbidLocalReferenceToLocal)

Tuesday, July 26, 2016 [Tweets]

Verizon Acquires Yahoo

Marissa Mayer:

This sale is not only an important step in our plan to unlock shareholder value for Yahoo, it is also a great opportunity for Yahoo to build further distribution and accelerate our work in mobile, video, native advertising, and social. As one of the largest wireless and cable companies in the world, Verizon opens the door to extensive distribution opportunities. With more than 100 million wireless customers, a shared view of the importance of mobile and video ad tech, a deep content focus through AOL, Verizon brings clear synergies to the table. And with their aggressive aims to grow global audience to 2B users and $20B in revenue within the mobile-media business by 2020, Yahoo’s products and brand will be central to achieving these goals. Joining forces with AOL and Verizon will help us achieve tremendous scale on mobile. Imagine the distribution challenges we will solve, the scale we will achieve, the products we will build, and the advertisers we will reach now with Mavens – it’s incredibly compelling.


The addition of Yahoo to Verizon and AOL will create one of the largest portfolios of owned and partnered global brands with extensive distribution capabilities. Combined, AOL and Yahoo will have more than 25 brands in its portfolio for continued investment and growth. Yahoo’s key assets include market-leading premium content brands in major categories including finance, news and sports, as well as one of the most popular email services globally with approximately 225 million monthly active users. Additional technology assets in the advertising space include Brightroll, a programmatic demand-side platform; Flurry, an independent mobile apps analytics service; and Gemini, a native and search advertising solution.

Justin Fox:

Most of that value -- Yahoo’s current market cap is $36 billion -- is in the form of Alibaba shares, but even the $4.83 billion that the operating business is selling for is, for a media company in these Dark Ages for media companies, not all that bad. The biggest newspaper company, Gannett, has a market capitalization of $1.76 billion; the biggest magazine publisher, Time Inc., $1.68 billion; the biggest owner of local TV stations, Sinclair Broadcast Group, $2.86 billion. The media and entertainment conglomerates that own cable channels, TV and movie studios, theme parks and the like are still worth a lot more than that -- but Yahoo never possessed those kinds of assets.

I am probably being too fatalistic here. Perhaps, with a relentless, visionary leader in charge all along (Jeff Bezos, who founded Amazon a few months after Yang and Filo put together their original Web directory, springs to mind), Yahoo could have found a path to continued relevance. Even simpler, it could have just bought Google: Semel offered $3 billion in the summer of 2002, but refused to raise the price when Google’s founders turned that down. Still, reinvention is something companies usually fail at when their business models stop working. Yahoo at least succeeded in preserving some value in the midst of decline.

Nick Heer:

Back in February of 2008, Microsoft offered $44.6 billion to acquire Yahoo, in an attempt to better compete with Google. At the time, analysts speculated that it would be a difficult merger for antitrust reasons — an idea that seems positively quaint today. In the end, Yahoo rejected the offer[…]

Tom Harrington:

I wonder if Verizon realizes that Yahoo! Is an implicitly unwrapped optional. There might not be anything there when they try to use it.

Maciej Cegłowski:

If you didn’t want Verizon to have your private data, you shouldn’t have joined Flickr in 2004. Think ahead!

Daniel Jalkut:

Flickr would be a boon to any team independently fired up to take on Twitter, Facebook/Instagram, Snapchat, etc., but lacking resources.

Update (2016-07-29): jelenawoehr (via Hacker News):

To truly understand how soul-crushing it was to work at Yahoo in pre-Marissa times, you should have been in Sunnyvale (I was, on a business trip) on the day Marissa announced free food in the company’s cafeterias. People scrambled to stuff themselves as if the announcement would be taken back in a day or two. The coffee shops were stripped of pastries. Yahoos packed multiple boxes at the salad bar and hoarded them in break room refrigerators. You’d think that the announcement Marissa made was a coming price increase for lunches, not free food. Good news at Yahoo was treated as suspect and likely to change at any minute.

Pretty soon, it was obvious that the new CEO was listening — really listening.


And about devel-random, or “d-r”: I was pretty active on that. No one told Marissa about d-r, but just a couple of days after she got there, on a Saturday IIRC, she responded to someone on d-r. This sent shockwaves throughout the upper echelons, and soon senior management were clamoring to get on d-r. Most of them dropped out, exhausted by the volume; but she stuck around.

Also: she used to use pine(1) to read her emails. That increased her stature in my eyes, and those of quite a few other engineers.


The ads will begin appearing on the platform starting today. Tumblr remains one of the most popular blogging platforms, attracting over 550 million monthly users to its blogs. Tumblr creators will have an opportunity to share in the revenue from ads on their blogs. The company says that bloggers will have the ability to opt out of the program should they wish not to participate.

Eleventh Hour Swift 3 Reviews

Erica Sadun:

It’s crunch time. Several reviews are running concurrently for just a couple of days. All breaking changes must wrap up in the next week and Swift 4 design starts on 8/1.


In contrast, SE-0132 is huge. It really deserves more time, attention, and consideration than the Apple-imposed deadline will allow. Brent’s done a lot of work on perfecting this. If time permitted, it would have benefited from a few more weeks of discussion and revision.


Swift 3’s timeline concerns me. It will be ready for GM release along with Xcode 8 and iOS 10. I, personally, would prefer Swift 3 to be delayed six months or more to get it right the first time.

The Novelty of the App

Walt Mossberg (Hacker News):

I attacked my phone’s app landfill to learn how very many apps which once seemed interesting or necessary hadn’t made it into the toolkit of my life. How many had been superseded by better apps or by functions built into the phone or my other devices since their debut. How many were redundant or disappointing. Or, on the other hand, how many were great but did more than I ever needed.


But there are way too many bad, mediocre, or me-too apps. The good ones are too hard to find. And the novelty and joy of suddenly having access to millions of bits of interesting software has worn off for me and for many other people. While Apple now claims 2 million apps in the App Store, as long as two years ago analysts were reporting that most smartphone users didn’t download even a single app in an average month.

Update (2016-07-26): Oluseyi Sonaiya:

Yes, there are application categories that are resistant to this disintermediation, but they are likely far narrower than you may assume, and likely represent a tiny sliver of the average smartphone user’s installed apps.

Swift Type Aliases

Ash Furrow:

After deleting the typealias and replacing it with a class of the same name, we didn’t have to worry about changing function definitions and property types all throughout our codebase. We made the changes locally, in one file, and most all the rest of our code still compiled. Pretty cool! Here’s the relevant portion of the pull request that made that change.


Any time you use the same tuple type more than once, consider making a typealias. In this case, the code became a lot shorter and easier to skim and understand.


Objective-C developers, burdened with arcane syntax for blocks, use C’s typedef to isolate that syntax strangeness in one place. And even though Swift’s closure syntax is awesome, we can still benefit from Objective-C’s example – we can use type aliases for closure signatures.


I recommend using a descriptive typealias that is private to your file, and then extending that typealias so you can keep things neat and tidy. […] We’re still extending the view controller, but specifically we’re extending the typealias so that the extension has a helpful name. This is another way that typealias can help add semantic meaning to your code.

Too bad Swift extensions can’t be named directly, like Objective-C categories.

Monday, July 25, 2016 [Tweets]

Microsoft’s Sweet Skype Solution

Richard Chirgwin (via AndrewMohawk):

In the same month Microsoft announced its alpha WebRTC-based Skype for Linux client, Redmond has put that native app and the native OS X Skype client on an end-of-life list.

This is because Skype is being rebuilt to replace its peer-to-peer architecture with cloud-centric code that supports Windows, iOS, Android and web browsers.

Microsoft has not given Mac and Linux users a date to dread. For now you can still get latest versions for all supported platforms – but it’s clear that full-fat Mac and Linux native clients aren’t something on which Microsoft wants to spend money forever.


The pace of change in our industry means that the devices and operating systems used by the majority of people shifts with time. Our commitment to deliver the best possible cross-platform experiences requires that we continually assess when it’s time to increase our focus on the platforms of the future. Sometimes this means that we must end support for some devices and operating systems.

Disable Find My Mac by Resetting NVRAM

Adam C. Engst:

There is one other problem that my friend Will Mayall alerted me to recently, which is that resetting NVRAM disables Find My Mac. Will discovered this on his own, but it turns out that others have run across the same fact over the past few years, as evidenced by a quick Google search. In essence, Apple stores the Find My Mac data in NVRAM, which is good for keeping it around even if the hard drive is removed, but bad in the sense that it’s easy to reset NVRAM — just restart while holding down Command-Option-P-R. A quick test confirmed the problem in OS X 10.11 El Capitan, and nothing has changed in the public beta of macOS 10.12 Sierra.

The only way to prevent Find My Mac from being disabled is to set a firmware password, which you must enter whenever you start up from a disk other than the usual startup disk. Plus, if you try to reset NVRAM, you’re prompted for the firmware password, and when you enter it, the Mac instead boots into Recovery mode. In fact, when you lock your Mac via Find My Mac, what it’s doing is setting a firmware password.

I still think Find My Mac is not worth the risks.

Swift Closure Capture Semantics

Olivier Halligon:

One important thing to note though is that in Swift the captured variables are evaluated at the closure execution’s time1. We could say that it captures the reference (or pointer) to the variable.

For those of you who know Objective-C, you can notice that the Swift behavior is unlike Objective-C’s default block semantics but instead somewhat like if the variable had the __block modifier in Objective-C.


To capture the value of a variable at the point of the closure’s creation (instead of a reference to the variable itself), you can use the [localVar = varToCapture] capture list.

Reversing the WWDC Wall

Martin Conte Mac Donell:

I took ~50 (rather sloppy) photos of the wall and wrote a program to do image stitching using a cylindrical projection.


With the flat image and using OpenCV again I detected all the contours (letters) and created a (inverse) mask to remove the blue-ish background and only keep the letters.


This image was already good enough to run tesseract. In order to get the best results I trained tesseract with the San Francisco Mono font before running it.


Go find your app phrase here: you can also get the JSON representation of the wall from here.

Sunday, July 24, 2016 [Tweets]

2016 MacBook – Two Months Later

Jeff Benjamin:

Although not everyone enjoys the low travel distance of the MacBook’s keys, it never posed a major issue for me. I have noticed, however, that some of the keys can get stuck as if infested with crumbs.


Jordan talked about the issues with USB-C in a prior post, and I can vouch for the fact that many of the USB-C peripherals that you’ll encounter can be finicky. I’ve experienced issues with devices abruptly disconnecting, devices that aren’t recognized without a reboot, and other annoyances.

Having a single USB-C port does occasionally present a problem, but that doesn’t bother me nearly as much as the wishy-washy interaction with some of my USB-C peripherals.

Previously: The 12-inch MacBook, Mistake One, The Developer’s MacBook.

Update (2016-07-24): Chris Turner:

I’ve had no keyboard issues, but fully concur on USB-C.

Sami Samhuri:

I bought the machine because I rarely use peripherals but have experienced USB disconnects with the battery I own.

But that is an exception. Since I don’t need max performance or use lots of peripherals, overall I love this MacBook.

Update (2016-07-25): Hampus Jakobsson (via Hacker News):

At the end of the day, I don’t think I am going back to my Mac. The ASUS [Chromebook] is super performant […] The biggest differences for me have been that there is never is a load time or memory running out.

Bridging Existentials & Generics in Swift 2

Benjamin Encz:

In an earlier blog post I pointed out some incompatibilities between type information that is statically known at compile time (Generics) and type information that is dynamically available at runtime (Existentials).


Given a heterogenous list of different instances that can be persisted we want to automatically find & call the DAO based on the type of object we encounter.


The .Self member, which would refer to the concrete type of the existential doesn’t exist in Swift 2. However, we can access the concrete type of the existential using Self from within protocols & protocol extensions.

Using a clever inversion of control we can use that Self type from within the PersistedType protocol (which all persisted types implement) to dynamically specify the generic type parameter of our GenericDAO<T>:

Exploring the App Store’s Top Grossing Chart

Graham Spencer:

One of the most striking things you’ll notice when browsing the Top 200 Grossing apps is that they are virtually all offered as free downloads. In my survey, just three apps were paid apps upfront; Minecraft (#33, $6.99), Grindr (#95, $0.99), and Facetune (#183, $3.99). The other 197 apps were free to download.


Whilst almost all the apps are free to download, it is also true that apps with In-App Purchases (IAPs) dominate the Top 200 Grossing charts. In my survey, just 2 apps did not offer any IAPs; Facetune (#183) and CBS (#200).


Games dominate the Top 200 Grossing charts, representing an overwhelming majority of 68% of the apps. The next closest is Social Networking at just 11% and comprised mainly of various dating apps.

Remote Code Execution With Image Files

CVE-2016-4631 (Hacker News):

An exploitable heap based buffer overflow exists in the handling of TIFF images on Apple OS X and iOS operating systems. A crafted TIFF document can lead to a heap based buffer overflow resulting in remote code execution. This vulnerability can be triggered via malicious web page, MMS message, iMessage or a file attachment delivered by other means when opened in applications using the Apple Image I/O API.


I was about to post that these exploits should be substantially mitigated by iOS sandboxing (you can get arbitrary code execution, but can’t get out of the exploited process’s sandbox without a second exploit), but then saw CVE-2016-4627 also in the 9.3.3 release notes, which is a local privilege escalation exploit that allows arbitrary code execution with kernel privileges.

It’s fixed in Mac OS X 10.11.6 and iOS 9.3.3.

Friday, July 22, 2016 [Tweets]

Xcode 8 Illegal Hard Links Prevent Cloning

After I installed Xcode 8 Beta 3, I could no longer back up my hard drive. SuperDuper reported errors like:

Error creating hard link /Volumes/HD Clone 14A/Applications/ to /Volumes/HD Clone 14A/Applications/ for inode (null)

Cloning HD, error creating hard link for file in

At first I thought that the drive was damaged, but the error still occurred after reformatting it. SuperDuper’s Dave Nanian:

Structure became illegal (hard link into app bundle) in 10.10.3 - break link by duplicating file...

I assume they just did this in the latest beta. (It was previously an issue with Retrospect & Anaconda.)

Indeed, does contain a hard-linked file:

ls -l /Applications/\
total 48
-rw-r--r--@ 2 mjt  staff   6141 Jul 14 18:08 AuditIssues.strings
-rw-r--r--@ 1 mjt  staff     42 Jul 14 18:08 Localizable.strings
-rw-r--r--@ 1 mjt  staff  10726 Jul 14 18:08 LocalizableOSX.strings

Note the “2” for “AuditIssues.strings”. Here are the two files with the same inode:

find /Applications/ -name AuditIssues.strings -print -exec stat -f "%i" {} \;

The fix is to make the two files independent copies:

cd "/Applications/"
mv AuditIssues.strings AuditIssues.strings.old
cp AuditIssues.strings.old AuditIssues.strings
rm AuditIssues.strings.old

Now the files have different inodes:

find /Applications/ -name AuditIssues.strings -print -exec stat -f "%i" {} \;

Update (2016-07-22): Mike Bombich (developer of Carbon Copy Cloner) on the similar issue with Anaconda:

This turns out to be an issue specific to *.app/Contents/PkgInfo and *.app/Contents/Resources/*.lproj files. OS X does not want to permit the creation of a hard link between one of these items in an application bundle to another file in a non-application-bundle folder. I was unable to find an explanation for this behavior in Apple’s documentation, nor in the source code for HFS or the OS X kernel.

Pixellating or Blurring Text Creates Identifiable Patterns

Kashmir Hill (via Nick Heer):

If you’ve ever pixelated an email address or blurred a phone number before putting an image onto the internet in order to protect someone’s privacy, I’ve got bad news for you: Researchers at the University of California-San Diego have found that the popular Photoshop redaction techniques are decodable such that the underlying text can be read.

The researchers were able to recover text from a variety of redacted screenshots that they found online, said computer science professor Hovav Shacham by email. They were, for example, able to figure out the blurred email address in this screenshot of a conversation between a corrupt DEA agent and the then-CEO of Bitcoin exchange Mt. Gox.

What Exactly Is “Compressed Memory”?

John Siracusa:

In Mavericks, the OS has one more option before it has to resort to swapping: compressed memory. Mavericks will find the least-recently-used data in memory and compress it, usually to about half its original size. Et voilà, more free memory.


Memory compression is a triple play for Mavericks. It’s a performance win; compressing and decompressing data in RAM is much faster than reading from and writing to disk, even an SSD. It’s an energy win; the less time spent moving data between RAM and disk, the more time the system can spend in its idle state. And finally, it’s a capability win; Mavericks can handle much more demanding workloads than previous versions of OS X before crying uncle.

This seems like a great feature, but I’ve never fully understood how it’s reported in Activity Monitor. What do the “Compressed Memory” numbers for each process and the “Compressed” total actually mean? This AskDifferent post lists the columns in Activity Monitor, but commenter James K Polk has the same questions as me:

If the activity monitor says a given process uses 621.4 MB of memory and 615.4 MB of compressed memory, does that mean that the process is really only using up 6 MB of memory? Or that 615.4 out of 621.4 MB was compressed down to some unspecified size? Or something else?

In other words, does a high number for Compressed mean that the compression has been effective or that the system is nearing capacity because most of what can be compressed already is?

It’s also not clear how compressed memory interacts with the other reported numbers. For example, why did Siracusa’s App Memory go down when the Compressed memory went up? I would have expected that App Memory would include the part that’s compressed.

And, presumably the memory that is paged out to disk is compressed. Does Swap Used take that into account?

Sandboxing Wisdom

Daniel Jalkut (tweet):

[I’ve] managed to produce two versions of my app, one of which causes the sandbox container to be apparently unwritable to the other after running! Specifically, preferences are not saved and console messages indicate an attempt to write preferences outside the host app’s sandbox.


These kinds of issues scare the bejeezus out of me because I really fret my users running into data migration problems after I ship an update, and because the relative opacity of the sandboxing system makes a lot of issues very hard to debug.

Daniel Jalkut:

In a nutshell: for the past 4 years or so, sandboxing has been a massive, amorphous “bug” that I have to wrap my head around. So, so tired.

Rarely a day goes by when I don’t worry that I made a huge mistake betting on sandboxing and MAS for the long run.

Peter Maurer:

Answer from a very tired sandbox wrangler: Don’t do it. Don’t waste your time on custom settings, etc. Instead, import once…

…via open dialog automatically, then make additional imports (for whatever reason) available via menu or preferences.

Ilja A. Iwas:

We released GarageSale 7 this week, still sandboxed, but won’t be submitted to the MAS. Ah, that sweet feeling of relief.

Daniel Jalkut:

This tool [asctl] appears to offer extensive insight into the sandbox’s understanding of containers. I wish I had discovered it earlier!

Apple doesn’t seem to have posted the asctl man page, but this online version has the same date stamp as the man page on my 10.11 system.

Thursday, July 21, 2016 [Tweets]

“This Regular Expression Has Been Replaced With a Substring Function”

Stack Exchange (Hacker News):

The direct cause was a malformed post that caused one of our regular expressions to consume high CPU on our web servers. The post was in the homepage list, and that caused the expensive regular expression to be called on each home page view. This caused the home page to stop responding fast enough. Since the home page is what our load balancer uses for the health check, the entire site became unavailable since the load balancer took the servers out of rotation.


The regular expression was: ^[\s\u200c]+|[\s\u200c]+$ Which is intended to trim unicode space from start and end of a line. A simplified version of the Regex that exposes the same issue would be \s+$ which to a human looks easy (“all the spaces at the end of the string”), but which means quite some work for a simple backtracking Regex engine.


This is not classic catastrophic backtracking (talk on backtracking) (performance is O(n²), not exponential, in length), but it was enough. This regular expression has been replaced with a substring function.

Tuesday, July 19, 2016 [Tweets]

AppleScriptObjC in Script Debugger 6

Shane Stanley:

The most important new AppleScriptObjC feature, and the most obvious, is how Script Debugger displays results. Instead of «class ocid»..., you will see much more.


First, although case matters in AppleScriptObjC, code completion is case-insensitive. You can type NSS or nss, and get the same result; the correct case will be inserted. You can also use the tab key within the completion list; this is very helpful where there are multiple methods with similar names.


Script Debugger includes terminology for the following frameworks: Foundation, AppKit, Quartz, AVFoundation, AddressBook, Contacts, CoreImage, CoreLocation, CoreWLAN, EventKit, JavaScriptCore, MapKit, OSAKit, and WebKit. The information is collated from the frameworks themselves and from header files, with some filtering applied to remove many methods that cannot be used from AppleScriptObjC.


Finally, there is an important limitation. Script Debugger runs scripts on a background thread, so any code that needs to be run on the main thread needs to use the method performSelectorOnMainThread:withObject:waitUntilDone: to do it. If not, you are likely to freeze or crash Script Debugger.

Previously: Script Debugger 6.


Scott Perry:

PSA: SQLite on Sierra/iOS 10 is built with SQLITE_ENABLE_SQLLOG, which makes it easy to create replayable SQL logs


This file contains experimental code used to record data from live SQLite applications that may be useful for offline analysis. Specifically, this module can be used to capture the following information:

  1. The initial contents of all database files opened by the application, and
  2. All SQL statements executed by the application.

The captured information can then be used to run (for example) performance analysis looking for slow queries or to look for optimization opportunities in either the application or in SQLite itself.


At runtime, logging is enabled by setting environment variable SQLITE_SQLLOG_DIR to the name of a directory in which to store logged data.

MacKeeper Threatens YouTube Video Maker

Mike Wuerthele:

Infamous software developer MacKeeper has demanded that four videos critical of its maligned tune-up utility suite be removed from the internet, threatening the teenager behind the videos with $60,000 in court costs and legal fees.


In August 2015, ZeoBIT, creators of MacKeeper, agreed to a settlement in 2015 to put $2 million in a compensation fund to cover attorney fees, refunds and administrative costs to U.S class action claimants who purchased MacKeeper prior to July 8, 2015.

The complaint alleged that “ZeoBIT intentionally designed MacKeeper to invariably and ominously report that a user’s Mac needs repair, and is at-risk due to harmful (but fabricated) errors, privacy threats, and other computer problems, regardless of the computer’s actual condition.” Court documents state that 513,330 people are registered owners of the software in the U.S.

Previously: MacKeeper.

Apple Music Learns From iTunes Match

Kirk McElhearn:

When Apple Music was released just over a year ago, Apple also debuted iCloud Music Library, a way of storing your iTunes library in the cloud. There were two ways to seed the cloud, either with iTunes Match or Apple Music. If you were an iTunes Match subscriber, matching your songs in your local library to your cloud library was done one way, and if you were just an Apple Music subscriber, matching was done differently.

This created some confusion about the way tracks were matched and stored in iCloud Music Library. Now, Apple is changing this, and will use the same matching method for both services. The company said that Apple Music now uses acoustic fingerprinting and provides matched files without digital rights management (DRM), or copy protection, just like iTunes Match.

Before, if you only had Apple Music, you only got the problematic metadata-based matching.

John Gruber:

I’m sure there are reasons for the way things are, but from the outside, combining iTunes Match and Apple Music should have been there from day one.

See also: Jim Dalrymple, iMore.

Monday, July 18, 2016 [Tweets]

The Strange Case of the System Preferences Window Width

Tim Schröder:

Apparently, the System Preferences window on my computer running OS X El Capitan was considerably wider than 668 (or even 595) pixels. Some research later it turned out that the width of the System Preferences window depends on the system language: The window will be 668 pixels wide when English is the primary language, but will have a different width for other languages. For example, with German set as primary language, as it is on my computer, the window is 762 pixels wide.


While Apple’s preference panes that come with OS X dynamically adjust their width to the actual width of the System Preferences window, this is not true for any custom preference pane, that will instead be displayed centered with more or less wide blank margins.

Medium URL Fragment Tracking

sime_vidas (via Ole Begemann):

Notice how this URL does not contain a hash (at the end). If you visit this URL, a hash will be added to the URL after the page loads. The hash value is some jumbled combination of letters and digits. What’s the purpose of this hash?


The idea is that when you share that page you will share it with the hash at the end. The visitors of the link you share will have their own hash generated to track their referrals. Medium will be able to see who visited the link that you shared, and build a tree of referrals (who the users you referred shared it with, who they shared it with etc). It’s a great metric to use in analytics when looking at user acquisition through social channels.


I hate this, I can’t tell you how many times I get a duplicate bookmark because I thought I didn’t have the article already in my bookmarks.

Exponential Time Complexity in the Swift Type Checker

Matt Gallagher:

But the line doesn’t get past the Swift type checker. Instead, it emits an error that the expression is too complex to solve. It doesn’t look complex, does it? It’s 5 integer literals, 4 addition operators, two negation operators and a binding to a Double type.

How can an expression containing just 12 entities be “too complex”?


If you don’t typically combine these features in your code, then you’re unlikely to see the “expression was too complex” error. However, if you are using these features, it isn’t always straightforward to suddenly stop. Mathematics code, large “function”-style expressions and declarative code are easier to write with these features and often require a complete rethink to avoid them.


A note about this approach though: unlike other languages, Double(x) is not equivalent to x as Double in Swift. The constructor works more like another function and since it has multiple overloads on its parameter, it actually introduces another overloaded function into the search space (albeit at a different location in the expression).


Posts like this: “[swift-dev] A type-checking performance case study”, indicate that the Swift developers believe resolving function overloads in the type checker is inherently exponential. Rather than redesigning the type checker to eliminate exponential complexity, they are redesigning the standard library to try and skirt around the issue.

Joe Groff:

Changing stdlib interfaces is done with an eye toward fixes like SE-0091 that mitigate inherently exponential work.

Most of the proposals around function labels and types are trying to kill type checker tech debt too.

“Rewrite the type checker” is up there on list of things we want to do.

Previously: Speeding Up Slow Swift Build Times, Swift 1.0 Performance and Compilation Times, Slow Swift Array Type Inference, Swift Type-checking Performance Case Study.

The Secret Life of Types in Swift

Slava Pestov:

I’m going to attempt to start by giving an overview of how types work in Swift, from the parser down to the lower layers of code generation in the frontend. Swift is a strong, statically-typed language with an advanced type system more reminisicent of functional languages such as OCaml and Haskell than something like C, so this seems like as good a place to start as any.


Types in Swift form a mini-language in of themselves, with a grammar consisting of nominal types as leaves, and structural types such as function types as interior nodes. Types are formed from TypeLocs and TypeReprs early on in semantic analysis. Further down in the compiler, sugar is removed and types are canonicalized, simplifying structural walks and equality comparisons. Substitution is a fundamental operation frequently used in the implementation of generics, and it is important to think about the role of types and declarations when performing substitutions for member access. Various higher-order operations simplify tedious boilerplate when manipulating types throughout the compiler.

Slava Pestov:

Now, let’s peel back a layer and dive into the type system of SIL, the Swift Intermediate Language. SIL adds a layer of detail missing from formal types, drawing a distinction between values and addresses, and making function types more explicit by introducing explicit annotations for argument and return value conventions.


At this point, we still cannot compile our code, but at least we can detect a type mismatch at the level of SILFunctionTypes, instead of just mis-compiling incorrect code. A situation where the formal types of the expressions match, but the lowered types do not is called an “abstraction difference”. Abstraction differences are handled by SILGen wrapping the substituted function value inside a re-abstraction thunk.

The re-abstraction thunk forwards arguments, calls the function, and forwards the result, taking care to handle any abstraction differences in the arguments and results. If the substituted argument is trivial but the original argument is passed indirectly, the thunk will load the value from its address and pass it to the substituted function. Similarly, if the substituted result is trivial but the original result is returned indirectly, the thunk takes the substituted result value, and stores it into the indirect return address given to the thunk.

Sunday, July 17, 2016 [Tweets]

Swift Classes to Be Non Publicly Subclassable by Default

SE-0117 (original, revision 1, revision 2):

The major observation here is that not all classes make sense to subclass, and it takes real thought and design work to make a class subclassable well. As such, being able to subclass a public class should be an additional “promise” beyond the class just being marked public. For example, one must consider the extension points that can be meaningfully overriden, and document the class invariants that need to be kept by any subclasses.

Beyond high level application and library design issues, the Swift 1 approach is also problematic for performance. It is commonly the case that many properties of a class are marked public, but doing this means that the compiler has to generate dynamic dispatch code for each property access. This is an unnecessary performance loss in the case when the property was never intended to be overridable, because accesses within the module cannot be devirtualized.

Chris Lattner:

As expected, this proposal was extremely polarizing, with valid arguments on both sides. The opinions held by supporters and opposers are held very strongly, and hundreds of emails were generated in a healthy debate about this topic.


On the first point, there are three related arguments against SE-0117:

First is that clients of Apple frameworks often choose to subclass classes that Apple publicly documents as being “not for subclassing”, as a way of “getting their job done,” typically as a way to work around claimed bugs in Apple frameworks. The core team and others at Apple feel that this argument is analogous to the argument that Swift should “support method swizzling by default”. Swift loves dynamic features, but has already taken a stance against unanticipated method swizzling, by requiring an API author to indicate where they allow method swizzling with the ‘dynamic’ keyword.

Yes, it’s absolutely analogous. It’s become obvious that “unanticipated” dynamism is not something that Apple wants to support with Swift. It’s clearly not Swifty, if by Swifty you mean the vision of the core team.

Second is that clients of some other public API vended by a non-Apple framework (e.g. a SwiftPM package) may end up in a situation where the framework author didn’t consider subclass-ability, but the client desires it. In this situation, the core team feels that a bigger problem happened: the vendor of the framework did not completely consider the use cases of the framework. This might have happened due to the framework not using sufficient black box unit testing, a failure of the imagination of the designer in terms of use cases, or because they have a bug in their framework that needs unanticipated subclass-ability in order to “get a job done”. Similar to the first point, the core team feels that the language is not the right place to solve this problem. Instead, there is a simple and general solution: communicate with the framework author and get them to add the capabilities that you desire.

Yes, there’s a bigger problem, and, yes, the language can’t “solve” this problem. That doesn’t imply that the language should forbid building a temporary workaround. In my view, the language should be helping to get the job done, not enforcing an idealized solution that may not even be possible because the vendor may be busy, uncooperative, or no longer available.

Third is a longer-term meta-concern, wherein a few people are concerned that future pure-Swift APIs will not consider subclass-ability in their design and will accidentally choose-by-omission to prevent subclass-ability on a future pure-Swift API (vended by Apple or otherwise). The core team feels that this is an extremely unlikely situation for several reasons. First of which is that it heavily overlaps the first two concerns. More significantly, any newly-designed and from-scratch APIs that are intended for Swift-only clients will make use of a breadth of abstractions supported by Swift—structs, enums, protocols, classes.

Indeed, this proposal is just one piece of a large trend. Swift is chipping away at dynamism for classes: subclassing, swizzling, hiding private classes and private methods. And the future, it appears, is APIs that rely more on structs, enums, and protocols, which are even more static. There are definitely some benefits to this direction, but I am concerned that it will lower app quality over the long term. This specific proposal may not have a huge impact because it affects neither Objective-C APIs (past) nor non-class Swift APIs (future).

To reiterate, as a summary, the core team agrees with conviction that it is the right default for public classes to be non-subclassable outside their module, unless they carry some additional indication by the API author that the class was designed to be subclassed.

Here’s my slightly exaggerated summary of the two points of view:

Me: I agree that dynamism shouldn’t be abused, but there are library bugs and limitations, and sometimes we need these tools to cope with them. I hate coding these workarounds and would love to remove them and unclutter my code as soon as the underlying issues are resolved. But bugs that affect the customer are much worse than temporary ugly code. Of course, ideally library vendors wouldn’t ship any bugs, but this is the real world. Nudging the API author to “think carefully” is well-intentioned but naive. The proposed solution is worse than the problem.

Apple: Making library vendors opt into dynamism will encourage them to think more about their designs. They will foresee how their APIs will be used. Making it impossible for developers to work around library bugs will foster communication with the vendor, who will then fix them. The fixing will be easier because the vendor won’t have to take into account developers’ workarounds. We don’t particularly care if your app is broken for months or years while waiting for a bug to be fixed. Developers won’t feel pressured to work around library bugs because competing apps will be subject to the same restrictions.

Here are some of the more interesting comments from the discussion:

Kevin Lundberg:

I do not want to be constrained by authors of libraries or frameworks into interacting with a system in only the ways they forsee. By making the default be non-subclassable, if a designer does not put thought into all the ways a class can be used then I as a consumer of the library am penalized.

Another point I can think of is mocking in tests. I’m unaware of any fully featured mocking solution in swift, and subclassing a class to override methods for mocking purposes is one way to make unit tests work with third party dependencies. If a class was not designed for subclassing, then writing tests where the class’s behavior needs to be suppressed would not be possible.


I’ve had issues before when working with third party C# code where the author didn’t declare a method as virtual, but the problem I was trying to solve required me to override that method. Luckily the code was open source, so I just embedded and edited the source into my project instead of using the prebuilt binaries, but that becomes a maintenance headache, and it is not possible for components that do not make the source available.

L. Mihalkovic:

Can’t really help for feel like it is training wheels all around… or padlocks on every kitchen cupboards.

Kevin Lundberg:

It’s not possible to accidentally override a method like it is in java or objective-c, so the proposal won’t help people who might accidentally override something. If a developer tries to override a method or class they probably have a specific reason in mind for doing so, and taking away the options that this ability presents, by nature of it being the default behavior, doesn’t sit well with me as it has the potential to take away a good tool from my toolbox.

I agree that API authors should have the power to restrict this dimension of a type’s usage, but I feel that it should be a conscious choice to do so.


I really enjoy designing elegant, well-defined APIs and this behaviour for classes has bothered me for some time.

On the other hand, I have also subclassed things that weren’t meant to be subclassed (e.g. I believe UITabBar was an awkward one back in the day, and UINavigationController has so many broken behaviours it’s almost a requirement to subclass and patch it, even though I believe Apple disapproves). I could fall back to Objective-C, but that’s an implementation detail. In some theoretical future with an all-Swift UIKit, what do I do about those issues?

After thinking about it, I decided that in this theoretical future, no App would be able to use those hacks, and so all Apps would have the same broken behaviours and degrade the quality of the platform to such an extent that Apple is forced to do something about it. Ultimately, that’s exactly what we want; they need to see broken Apps everywhere in order to prioritise fixes. That improves code quality all-around.

Tino Heth:

I can’t fight the feeling that many fans of ideas like this believe that good designed libraries come for free by simply adding restrictions — which, at least in my opinion, just isn’t true: No matter what the defaults are, good libraries are hard to build, so I predict this proposal would not only fail in increasing framework quality, but also will make it much harder for users of those frameworks to work around their flaws, which are just a natural part of every software.

Michael Peternell:

I think it’s a step backwards. […] It’s not bad to allow sealing of classes. I don’t see real value in it though. And “sealed” should not be the default. Either the class if final or not: for me that’s a valuable distinction. “sealed vs. unsealed” is not.

Paul Cantrell:

In all the OO languages I’ve used, classes are open by default. That raises a question: is this really a good idea? Aren’t we ignoring established precedent? What about all the unauthorized subclassing we’ve done in the past?

I’m sympathetic to those accustomed to Objective-C, Ruby, Javascript, Python, and other languages with more fungible type systems who argue that it’s unreasonable to expect us all to use classes only as library authors intend. Monkey patching has saved many a day, they’d say. And it’s true, it has! I rely on it.

My counterargument: the ecosystem is changing.


The big exception to this is Apple itself, and the direction of this proposal implies a large cultural shift in how Apple deals with its developer community. Having the Swift language so aggressively prevent the dubious, brittle, bad-idea-but-it-gets-the-job-done workarounds that the Obj-C runtime allowed means that Apple’s platforms are going to have to be more consciously extensible, more transparent in their design decisions, and more responsive to unanticipated needs.

Tino Heth:

With “sealed by default”, the situation changes: Users are protected from some simple bugs, but nonetheless, they’ll encounter situations where the library doesn’t do exactly what they want. So, you take a look at the source, find the problem, and fix it. It’s no bug at all, it’s just a tiny degree of freedom that is missing because it wasn’t important to the author. You can create a pull request as well, but it doesn’t offer a real improvement to the author, who is already busy with dozens of similar requests by other users -> you end up with a custom branch with all the housekeeping associated with it.

So overall, I’m quite sure that the proposal won’t improve software quality, but rather degrade it.

Ricardo Parada:

I am afraid that developers will not take the time to specify which methods are overridable resulting in libraries that are difficult to patch, extend.

In my 26+ years of object-oriented design and programming (other languages, Objective-C since 1990 and Java since 2001) I have worked with object oriented libraries and subclassed methods that the authors probably never anticipated. I have been able to fix problems, enhance classes by creating subclasses with fixes and enhanced behavior.

In java for example I have seen that sometimes I would have been able to fix bugs or enhance the existing classes had the author not chosen a method to be protected or private. Sometimes they had a good reason but sometimes they didn’t. Is have been able to survive using an awesome library that was discontinued and end-of-lifed thanks to subclassing that has allowed me to fix problems and enhance over the years as the Java language kept evolving.

Jakub Suder:

I agree that subclassing library classes that were not explicitly planned to be subclassed might sometimes lead to bugs. But just because a technique or feature of a language allows you to sometimes make mistakes, it doesn’t mean it should be completely disallowed. By this logic, we’d have to remove all instances of "!" any anything related to pointers from Swift. It’s possible for a feature to allow you to shoot yourself in the foot, and still be useful at the same time.


I don’t think the problem is significant, and I’m afraid the proposed solution is worse than the problem itself.

Jonathan Hull:

I don’t think this proposal will achieve its stated objective of forcing people to think about subclassing more. It will just add confusing boilerplate.

Things like Swift optionals work well because they make the (often forgotten) choices explicit in the context that they are used. In the world of Human Factors, we call it a forcing function. This proposal has the inverse structure, and will be ineffective, because the “forcing” part of it shows up in a different context (i.e. trying to use a framework) than the decision is being made in (writing the framework). This type of thinking leads to things like Java and the DMV.


Those who were prone to be thoughtful about their design would have been anyway. Those who are not thoughtful about their design will just leave these annotations off… leaving us with no recourse to extend/modify classes. When people complain, they will add the annotations without actually thinking about the meaning (i.e. stack overflow / the fixit tells me I need to add this word to make the compiler happy). All this does is put framework users at the mercy of the framework writers.

Brad Hilton:

I […] give a strong, strong, strong -1 to this proposal. To make classes non-subclassable by default is only going to lead to unanticipated pain and frustration. Also agree with other comments that subclassable and overridable conflate access control with class behavior. If we want to make it possible to define a class as non-subclassable to external users, I’d agree to something more consistent with existing swift access control like public(final) as has been proposed by other commenters. However, I agree that making classes final by default is a bad idea that will create a larger problem that it solves.

Jean-Daniel Dupas:

I can’t count the number of times it save my hours [to be] able to override arbitrary classes and methods.

Sometimes to simply add log point to understand how the API work. Other times to workaround bugs in the library. Or even to extends the library in a way that the author did not intent in the first place, but that was perfectly supported anyway.

I already see how libraries author will react to that new default. They will either don’t care and mark all classes as subclassable, or find to burdensome to get subclassability right and prohibit subclassing all classes.

Jean-Daniel Dupas:

Nonetheless, being able to subclass any class allowed me to solve a bunch of very bad memory leaks in the new NSCollectionView implementation, and made it usable for my project.

Thinking than you will have to work only with perfectly design libraries without any bug is utopian. And thinking you can predict every usages of your library by users is short sighted IMHO.

John McCall:

I sympathize with the argument about wanting to fix bugs and add features via override, but that’s never been maintainable in the long term; you always just end up with superclasses that everyone is terrified to touch because every subclass has its own invasive “fixes”, and that’s even when working within a single codebase. With libraries, you can pretty quickly get locked in to a specific version because your customizations don’t work with new releases; either that, or the maintainer just decides that they can’t fix of their mistakes and so goes off to rewrite it from scratch. Either way, it’s not good for the ecosystem.

Jeremy Pereira:

Thirdly, this is a huge breaking change. Not only is it a breaking change, but for a lot of people, the breakages will be outside of their control. Consider if I publish a module with a class with public methods and you subclass it in your code. Once this change is implemented, my code will still compile and pass its unit tests but your code is now broken and you are dependent on me changing my code to fix your code.

Tino Heth:

Defaults matter, because they transmit a message:

Every rule and obstacle we add to Swift is a statement that says “we favor bureaucracy over freedom”, and this will affect the community evolving around the language.

When you use a library in a way that wasn’t anticipated by its author, you’ll ran into issues occasionally; nonetheless, I think we should struggle for an open ecosystem that encourages others to experiment and fail, rather than to limit their possibilities in a futile attempt to protect them.

Garth Snyder:

I’ll just note that this proposal reminds me of the fortune file entry, “Whenever you see a sign that says ‘no exit’, it means there is an exit there.”

Specifically, under this proposal, whenever you see a nonsubclassable, non-final class (which I suppose would in fact be any class not explicitly marked as subclassable), it would mean that the library implementor IS in fact subclassing it. Otherwise, it would just be final. Therefore, it’s by definition a perfectly reasonable class to specialize. The implementor just doesn’t want YOU doing it.

Maybe the implementor has a very good reason for this. But in the modal case, probably not.


There should absolutely be a way to say “don’t subclass this.” But do you really need anything more than a comment that says “don’t subclass this?” Perhaps as a compromise, and as a concession to the value of automated checking, this feature could be reformulated as a warning system. A class marked as nonsubclassable could be extended only be including a corresponding acknowledgment keyword in the extending class, a kind of Swift version of -IHaveBeenWarnedThatAPFSIsPreReleaseAndThatIMayLoseData.

Paul Norton:

Because of that experience, I am fully convinced that library authors will never imagine all the ways developers will use a library’s API.


I also think it would eventually force a second change; some sort of override to the default, at which point we’ve extended the language further to get closer to where we started.


Personally, Im not against sealed by default, but I think there are cases where closed source libraries have certain cases where workarounds are necessary, and just sealing by default will prevent those cases.

One could say, “well just use an open source one, or change vendors” but its not that easy in The Real World™ where we get certain SDKs shoved down our throats by the suits… and while that may be a separate issue to the one at hand, its still a problem that won’t resolve itself by simply locking down things…

In my own case, Ive fought with NSBrowser / NSTreeController in the past and the only way to resolve things was to subclass (and no, waiting 1 or 2 years for a fix is not acceptable if you already have a product in the wild).

Jonathan Hull:

Please stop saying that this proposal will bring more consideration to the design of libraries. It isn’t true. I haven’t even seen an argument for why it would be true, it is just taken for granted that it is true.


It sounds good, but at the end of the day, people are human and they will make mistakes. Best to either catch those mistakes in the context where they happen or to mitigate the effect of it. This proposal basically forces you to feel the full effect of other people’s mistakes (thinking that it will discourage them from making them in the first place).


The current proposal will only cause massive problems down the line, IMHO. We will find an escape hatch is needed, but we will have made optimizations based on assumptions of finality which prevent us from easily adding one.

L. Mihalkovic:

There are IMO no advantages to sealing by default. If there were, I cannot imagine how they can have so consistently eluded the designers and maintainers of so many great OOD languages in the last 30 years. Does it mean it is just a matter of time for the core team to take it the c++ standardization committee to make sure C++ gets enhanced the same way?

L. Mihalkovic:

After 30 years of Objc’s msg dispatching being touted as not a performance problem, all we hear for the last couple WWDC (did you pay attention to the ‘they are getting it’ during one of the session) is how much every single dynamic dispatch must be chased out of our systems, culminating with this ‘seal by default is best for you’. Why can’t developers [writing] code be made responsible for writing well or not?

Jonathan Hull:

There is also a dangerous difference between helping the programmer catch mistakes (e.g. don’t accidentally subclass the wrong method) and trying to prevent them from coding in a style you disagree with. I have been seeing far to many proposals of the second variety of late.

let var go:

It is the opposite of nearly every other OOP language’s behavior and the opposite of what anyone will expect who is coming to Swift after doing any OOP programming anywhere else. While I believe that Swift should be free to buck the trends of other languages where there are significant advantages, changing something this fundamental will introduce a new learning curve for even experienced programmers. The advantages better be significant, otherwise you are introducing a frustrating quirk into the language without any great benefit in exchange. But the advantages of default non-subclassability are marginal at best.


The developers who don’t make careful design decisions will just go with the default - in this case they will just leave the default in place, and their public classes will not be subclassable. That doesn’t improve anything, it just makes sloppy, poorly written code harder to fix in the off-chance that you are stuck working with it. In other words, quality will not improve, but productivity will suffer, because it will be harder to develop workarounds for the problems that will inevitably appear in even the best-designed APIs.


But this whole issue of default non-subclassability doesn’t fall into that category of “safety” at all. It doesn’t help a programmer produce safer code. The same hidden flaws will persist whether non-subclassability is the default or not. The difference is that those same hidden flaws will be more difficult to deal with after the fact.

Colin Cornaby:

The “final should be default because adding final after the fact is destructive” arguments are interesting, but ultimately not convincing to me. If you thought your class was safe for subclassing, but it ultimately wasn’t, this solves nothing. You’ll mark your class as subclassable, and you will ship it, and there will be issues, but it will still be too late to take things back. If you know your class is not safe for subclassing, you should mark it as final. There is no advantage here in that scenario.

This is all part of building a safe public API. Public API design can be difficult, but if you don’t understand safe subclassing designs, you are likely to miss the issues and mark your class as subclassable and ship it anyway. Again, the best way to tackle this is to find better ways to surface the issues. Making final the default still doesn’t solve the core issues of people not understanding the right design.

Jonathan Hull:

My point is that you are completely ignoring an entire class of risk that has a real-world $$$ cost. Every time I have to use a framework under this proposal, I am now completely at the mercy of the author. In the case of open source frameworks I can at least make a fork, but for closed source frameworks (often from large companies where us using their framework has been negotiated by the bosses) you have now just added weeks to my development cycle while I wait for big-company-who-doesn’t-really-care-that-much to update their stuff.


Are you offering to explain to my boss/client why I can’t add the feature in a reasonable timeframe like I can with Objective C frameworks? That it may not even be possible now in Swift even though the Android guy just did it in a few hours?

Do you know what I am going to tell my boss/client? “Don’t use Swift for frameworks” and “Try to avoid partnering with companies that have Swift frameworks”. “It is too much of a risk”. “We are giving them too much control over the future of our product…” I mean, it even affects the prices that companies can charge for crappy frameworks. If I am asking for a bunch of features that I need them to add to provide a basic feature, that affects negotiations/price (vs a world where I can add it myself if needed). Sealed-by-default gives them leverage.

Tony Allevato:

The argument shouldn’t be “we need open subclassing or we can’t fix broken APIs”; that’s a false choice. It should be “we need something to fix broken APIs”, and that “something” should be proposed and the need for it should be argued in its own right.

David Sweeris:

-1 for this proposal, but +1 for solving the issues it raises

Regardless of what ends up being the defaults, I’m a very strong -1 on conflating visibility and subclassability/extendability.

Jon Akhtar:

I completely agree, the programmer should have as much power as we can give them, even if it means allowing them to shoot themselves in the foot, because at the end of the day Swift isn’t an academic exercise, it is a real world programming language, so it should be optimized for solving real world problems not having some kind of technical, philosophical, and stylistic perfection when those come at the cost of the former.

Update (2016-07-17): Daniel Jalkut:

It’s absurd to imagine you can predict all valid subclassing patterns. […] Apple, nor any framework provider, can provide functionality sufficient to cover the breadth of innovation. Clever hacks built the industry.

To be clear, although the proposal was “returned for revision,” it is essentially accepted. It’s just the syntax that’s being debated now (e.g. changing subclassable to open).

Update (2016-07-18): Friedrich Markgraf:

Same issue with Apple’s removal of Xcode plug-ins it prevents unanticipated innovation

Marco Scheurer:

Another nail in the coffin. Swift: “No programmers except myself should be trusted”

Daniel Broad:

If objective-c had been like this we would have had only default coloured navigation bars until ios4.

Casey Liss:

I’m… not keen on the idea of final by default.

Do not want.

Ilja A. Iwas:

This is why you don’t let compiler engineers control the fate of your computing platform(s)

Orta Therox:

This feels like a real step against getting things done in Swift

Nick Lockwood:

I’m in two minds. It won’t impact existing Cocoa APIs as they’re all ObjC, & future Swifty APIs will use fewer classes anyway.

Radek Pietruszewski:

But values are never the stuff in need of hacking, it’s the classes that do complex stuff.

Mike Ash:

This “claimed bugs” language makes me uncomfortable.

Adam R. Maxwell:

I’ve been worried that guys designing languages/frameworks don’t maintain GUI apps using them, or they’d know there are bugs.

Florent Pillet:

I’ve been fighting with lack of dynamism on Android. When you find issues in the OS you have to rewrite the whole class

Paul Haddad:

My outsider’s perspective of Swift: X can be misused, so we’re banning X.

Seems like coding with safety scissors.

Robert Cottrell:

It seems like a lot of recent decisions are justified “it’s no so bad because you can just go and ask the vendor to fix it”.

Leon Breedt:

yes Radar is a fantastic bidirectional communication tool with a responsive vendor, what could go wrong?

Nacho Soto:

if Apple knows there’s no workaround for their bugs, maybe they’ll get better at shipping quality software.

Max Seelemann:

Well, apparently the core team never had to write software against Apple frameworks. They have no idea how buggy they are.

Friedrich Markgraf:

They just bring their stuff over to Craig’s house if it doesn’t work. They have a mechanism for fixes; we don’t.

Jeremy Tregunna:

This will raise the cost of developing software, and make bugs last longer. That’s the issue at stake here.

Or have you ever been at the mercy of a non-active / committed library author? :)

Krzysztof Zabłocki:

A lot of cool things I built for the community or in my own apps would be impossible to do if this happens.

Adam Kaump:

+ Encourages more thoughtful API design

+ Encourages community-driven changes to frameworks that can benefit everyone

Sean Reilly:

:( re Apple’s stance on the Swift final-by-default proposal.

Mike Ash:

“I think so, Brain, but don’t people need to ship software without waiting years for us to fix the bugs they encounter?”

Steven L:

for final in swift, is there no backdoor option? ’cuz if it’s truly enforced, that’s super painful and short-sighted


I think this is one of the worst decisions in the history of language design.

Scott James Remnant:

I disagree that an open method overridden from a superclass is implicitly open.

As the rationale for the proposal states, overridability is hard to get right, and there is no guarantee that the consumer of an API is going to think about it. The default for override methods should not be open or final, it should be the internal equivalent.

John McCall:

I don’t think that defaulting to non-open would be a good idea. Like I covered in the proposal, inherited open methods remain open; letting an override implicitly close off an open method would create a pretty unfortunate error-of-omission situation.

We could remove the default here and require the method to be explicitly open/nonopen/final, but then we really do pile up the modifiers[…]

L. Mihalkovic:

So simply as a matter of following the logic of the proposal, every subsequent extension of something explicitely marked as open for subclassing should defacto be placed back into the default non-subclassable state, forcing the author to have to explicitely validate the new set of relationships existing between the methods in the subclass. Otherwise the logic does not hold, and the original proposal does not hold water[…]

Károly Lőrentey:

I’m enthusiastic about sealed-by-default classes, but to be honest, I personally have no idea what default (if any) would be best for class members. Ask me again after I’ve worked with the new classes for a couple of months.

L. Mihalkovic:

This is what i find so unique about this situation: there is not 2 months to decide, there is not a couple of implementations to compare.. there is here and now to decide for the next 20 years, with zero experience with the api and very little external references (which nobody seems to [have] any practical experience with) ... nonetheless it must all be fleshed out before the 28th. I [sincerely] hope the core team is more prepared than they currently let out.

John Randolph:

- infinity for this proposal.

If adopted, it will mean that whenever I want to extend a class implemented by a short-sighted developer, I’ll have to resort to wrappers and classes that own instances of the lobotomized class, etc, etc.

Joe Groff:

Users who are third-party framework devs I expect will benefit the most.

They don’t have Apple’s resources to deal with all the backward-compat testing to cope with unconstrained subclassing and patching.

Jonathan Hull:

I would really like to see the methods match the open-ness or final-ity of their enclosing scope by default. Note: I also feel this way about access levels, which work this way except for public… I believe they should work the same way for public too (while keeping internal as the default level for top-level structures). If a class is public open, I am typically going to want 85%+ of the methods to also be public open. Having to annotate each of them adds a lot of visual noise, and is fairly easy to forget to add when refactoring amidst that visual noise. I already have this problem with public, so I expect it will only be worse for public open.

Update (2016-07-19): @Gary_BBGames:

I like Swift, but this will be an absolute shit sandwich

Gwynne Raskind:

The reasoning from the core team that working with vendors replaces the ability to subclass to work around problems simply doesn’t hold water for me. This is a very real and very common issue, and there are endless cases where vendors won’t or even can’t solve the problem, especially in closed-source code.


In short, saying “filing a bug will work” isn’t good enough to justify locking out the ability of developers to deal with problems in vendor code. It’s simply not true - it’s certainly almost never been true of Apple frameworks, and even when it has, that doesn’t help anyone “now”. (To be clear, I’m not suggesting Apple is unresponsive to Radars. However, I am saying that there’s no transparency, no confidence in getting fixes, and no hope of any kind of reasonable (from a local perspective) timeline for deployment of fixes.)


If not for the practical considerations, I’d love it! Conceptually speaking, I find it elegant, even harmonious. But again, in practice, the result isn’t so pretty.

Juan Laube:

-1, I really think this a step in the wrong direction.

I recognise the problem around this, and why something is needed. However, I don’t like the idea of restricting things by default. In the attempt to solve a problem, we will create more problems by introducing more workarounds to replace what now is being overridden.

Brent Royal-Gordon:

You know, one thing I haven’t seen mentioned is that, just as sealed-by-default preserves the options of library programmers, it also preserves the options of the language itself.

Suppose the people who think this is a huge mistake are correct, and we ultimately conclude that sealed-by-default is a disaster. What can we do about it? Well, we change Swift 3+n to open all classes by default. This would be source- and binary-compatible with all existing code; the only wrinkle is that classes compiled with a sealed-by-default compiler would still be sealed. (And that’s not even a problem yet, since stable ABIs are not a thing yet.)

The reverse, however, is not true. Going from open-by-default to sealed-by-default is source- and binary-incompatible. If we don’t do it now, we may never be able to do it.

Tino Heth:

In reality, nobody will measure the effect of this change, and those in favor of the proposal might even call the absent of a horrible catastrophe a “proof” for their opinion.

David Owens II:

As an API author, if I need to have a value type, I cannot use inheritance. And more important, especially with regards to many of the arguments against this proposal, you cannot fix any issues with these struct types with inheritance either. It’s at this point that I find your arguments extremely weak: if it is so crucial that inheritance be enabled by default for class types, why are you not more concerned about Swift’s focus on value types and using value semantics for APIs? After all, there are pretty much no APIs in Swift’s libraries that you will be able to patch this way.

I am, and I’ve been writing about that concern from the beginning. As I said at the top, this proposal is just one piece of the larger trend.

There’s absolutely nothing in this proposal that prevents Swift from providing tools to get you access to what you need. For example, imagine a world where you could download a developer Swift module that contains all of the unoptimized code.

This is the hypothetical “escape hatch” that people have been talking about. In theory it could work, if implemented, and if you were able to get all the unoptimized code. That seems rather unlikely to me, and it would be a lot of trouble to go through for the (I think) small benefits the proposal would provide. There are so many more pressing issues I would like the Swift team to work on than building restrictions and then an escape hatch for them.

Update (2016-07-21): Benjamin Mayo:

In general, language design should not be decided by the possible existence of buggy code. However much we strive to make perfect code, there will always be bugs.

Eli Perkins (via Natasha Murashev):

I would have loved to seen this implemented as an opt-in compiler-time flag, rather than a syntactical level keyword.

Peter Livesey:

It seems like for 3rd party libraries only, we are afraid that people will subclass things when they shouldn’t? Is this a real problem? I’ve honestly never seen anyone incorrectly subclass something in a 3rd party library when there was a good alternative. Maybe I have and I forget, but it happens so rarely that I don’t think it’s a real problem to fix.

Peter Livesey:

With this proposal: consumers are screwed until the library is fixed, but the change is backwards compatible.

Without this proposal: consumers are fine to choose what to do (and maybe make mistakes), but the change could be backwards incompatible.

To me, consumers not being screwed > changes being backwards compatible.

Tal Atlas:

I have no concerns intrinsically to the behavior discussed but rather to the complexity that this brings. Creating another type of access control and possibly keyword just adds complexity to an already fairly complex language.

Nevin Brackett-Rozinsky:

I am reminded of the atomic / nonatomic situation in Objective-C. Yes, atomic is generally safer, but almost every property in almost every class is declared nonatomic. It was a mistake to set atomic as the default, which caused a profusion of noise in property declarations.

Chris Lattner:

The proposal has been returned for revision, again. :-)

As with the first round of discussion, the community generated a large number of emails, exploring the various aspects of the proposal. While many community members agree with the thrust of the proposal, a number of people are concerned with the boilerplate being introduced by the proposal, among other issues. The core team spent over two and a half hours discussing this topic from first principles, and has come up with a similar-but-different approach that should reduce the boilerplate, while still accomplishing the primary aims of the proposal. John McCall will be revising the proposal today and we’ll restart a short discussion period about it tomorrow.


Conflation of public and open; it feels like open is a new higher access level, like getting married, or going sudo or something. If this proposal was accepted, open should substitute public, and never be alongside it (the same way public private class makes no sense)

Garth Snyder:

In the original proposal (and the ensuing discussion), there was tacit agreement that subclassability/overridability and access levels should be orthogonal. However, given the direction that the design has taken since then, I think we should revisit that decision.


Third, developers already understand access levels and how they interact. If open is just an access level, all of this proposal’s changes can be fully and naturally described in one line: “public no longer includes the right to subclass or override. To get the behavior formerly known as public, use open instead.” Clear, concise, and not very controversial.

Update (2016-07-23): See also: Core Intuition.

Update (2016-07-25): Scott James Remnant:

I have no objection to the fundamental concept of the proposal, but this should not come at the cost of the language, just on the grounds of the timeline. I think it would be better for this to be re-proposed without the rush for future versions of Swift - especially since it’s “additive.”

Dmitriy Y. Volkov:

I’ve been reading mailing list for the last two reviews of this proposal and discussion turned from “We shouldn’t add this at all” - which was kind of justified to “How actually will it interops with other swift features” - which still has a lot of questions of how exactly this modifiers will play out with other accessibility modifiers, and it seems like there is still no single answer. As Scott said, I have the same feeling that this proposal is being rushed to be accepted before the changed to Swift 3 are locked, and this won’t do any good to its semantics.

Gwynne Raskind:

This is one of the most contentious topics I’ve seen come across Swift so far. While there may be no stopping the avalanche now as far as implementing the proposal in general (which I remain against, though many of the arguments I’ve heard in favor of it are starting to shake my conviction!), it definitely begs more consideration than running up hard against a release deadline. Goes double considering the disagreement even among the proposal’s supporters about the best semantics.

Update (2016-07-29): Chris Lattner:

The third review of “SE-0177: Allow distinguishing between public access and public overridability” ran from Active review July 21...25. The proposal has been accepted with revisions.

This proposal was far better received by the community than previous versions of the proposal, and the “first design” was the favored path within it. However, there were some concerns raised about the complexity of the model, stemming from non-obvious combinations like “open private”. As such, the core team has requested that the proposal be revised to make “open” function as another access control specifier. “open” is now simply “more public than public”, providing a very simple and clean model.

John has already revised the proposal to the new model, I encourage you to read it if you haven’t already.

Here’s the final version of the proposal.

Friday, July 15, 2016 [Tweets]

Lepton Image Compression

Dropbox (Slashdot):

We are pleased to announce the open source release of Lepton, our new streaming image compression format, under the Apache license.

Lepton achieves a 22% savings reduction for existing JPEG images, by predicting coefficients in JPEG blocks and feeding those predictions as context into an arithmetic coder. Lepton preserves the original file bit-for-bit perfectly. It compresses JPEG files at a rate of 5 megabytes per second and decodes them back to the original bits at 15 megabytes per second, securely, deterministically, and in under 24 megabytes of memory.

We have used Lepton to encode 16 billion images saved to Dropbox, and are rapidly recoding our older images. Lepton has already saved Dropbox multiple petabytes of space.

See also: StuffIt X.

Improving Color on the Web

Dean Jackson:

The past few years have seen a dramatic improvement in display technology. First it was the upgrade to higher-resolution screens, starting with mobile devices and then desktops and laptops. Web developers had to understand high-DPI and know how to implement page designs that used this extra resolution. The next revolutionary improvement in displays is happening now: better color reproduction. Here I’ll explain what that means, and how you, the Web developer, can detect such displays and provide a better experience for your users.


The Web has often struggled to handle colors correctly. I’m sure there are some readers out there who painfully remember Web-safe colors! While we’ve moved on from that, we still have limitations, such as HTML and CSS having been defined to work only in the sRGB color space. Just like the example of hober’s shoes above, this means there are many colors that your CSS, images, and canvas are unable to represent.


WebKit now supports the (new to CSS Color Level 4) color-gamut media query.


This is what members of the WebKit project have proposed for CSS. The current idea is to add a new function called color() that can take a color profile as well as the parameters defining the color.

Gus Mueller:

This writeup is an incredibly great explanation of wide gamut issues with a touch on deep color as well. If you are a developer who uses color in any way, you’re going to want to read this. Wide gamut displays are already here.

Previously: iPad Pro, True Tone, and Color Gamut.