Archive for July 2016

Saturday, July 30, 2016

Looking Back on Swift 3 and Ahead to Swift 4

Chris Lattner:

From our experience with Swift 3, we know we need to pick and choose what we’re going to tackle. For Swift 4, the primary goals are to deliver on the promise of source stability from 3.0 on, and to provide ABI stability for the standard library.


String is one of the most important fundamental types in the language. The standard library leads have numerous ideas of how to improve the programming model for it, without jeopardizing the goals of providing a unicode-correct-by-default model. Our goal is to be better at string processing than Perl!


Adding an (opt-in) Cyclone/Rust inspired memory ownership model to Swift is highly desired by systems programmers and folks who want predictable and deterministic performance (for example, in real time audio processing code). More pertinent to the goals of Swift 4, this feature is important because it fundamentally shapes the ABI. It informs code generation for “inout”, how low-level “addressors” work in the ABI, impacts the Swift runtime, and will have a significant impact on the type system and name mangling.


The core team is committed to adding powerful dynamic features to Swift. For example, Swift 3 already added nearly all the infrastructure for data reflection (which is already used by the Xcode memory debugger). We should use this infrastructure to build out a powerful user-facing API. Similarly, we would like to design and build out the implementation for dynamic method reflection runtime + API support.

Update (2016-07-30): See also the ArsTechnica and Hacker News comments.

Chris Lattner:

You can choose to use the memory ownership features by adding extra annotations, giving better performance and control over ARC. Right now we have very limited options for avoiding ARC overhead in critical loops, largely forcing you to drop down to unsafe constructs. We’d prefer the model to be “you can add more annotations to your code to get better performance, allowing the compiler statically verify correctness instead of dynamically”.

Memory ownership control is an extremely non-trivial feature, which will probably drive us to add first class move semantics and region types to the language. This will also call for significant standard library extensions. It will pay for this complexity by making it easy to ignore the complexity if you don’t want it, and by the fact that the standard library and other stuff can go much faster.

Friday, July 29, 2016

Developer Tools Survey

Dave DeLong:

Survey: what non-apple dev tools are you still using, 6+ months after you first got them?

I’m currently using: BBEdit, FogBugz (in Google Chrome), Git and Tower, LaunchBar, Dash, make, Python, VMware Fusion, Hopper, Kaleidoscope, Hex Fiend, xScope, Deploymate, RB App Checker Lite, Script Debugger, SourceTree, Base, Paw, Soulver, and OmniFocus. I plan to switch from Xcode bots to a third-party continuous integration solution.

For the non-code parts of my apps, I also use reStructuredText, pdfLaTeX, Counterparts Lite, Acorn, Icon Slate, and Opacity.

And for deployment: DropDMG and rsync.

Update (2016-08-06): Vadim:

So Dash is the absolute champion!

Python Internals: PyObject

Sergei Danielian (via Hacker News):

Each Python’s type implementation (PyIntObject, PyFloatObject or PyDictObject) has PyObject_HEAD located as its first member (or the first member of its first member, and so on). This member sub-object is guaranteed to be located at the same address as the full object.

The PyObject_HEAD refers at that member sub-object, but could be cast to the full type once ob_type has been inspected to get knowledge of what the full type is.

Philip Guo:

Here are nine lectures walking through the internals of CPython, the canonical Python interpreter implemented in C. They were from a dynamic programming languages course that I taught in Fall 2014 at the University of Rochester. The format isn’t ideal, but I haven’t seen this level of detail about CPython presented online, so I wanted to share these videos.

Jake Vanderplas:

We saw above the extra type info layer when moving from a C integer to a Python integer. Now imagine you have many such integers and want to do some sort of batch operation on them. In Python you might use the standard List object, while in C you would likely use some sort of buffer-based array.

A NumPy array in its simplest form is a Python object build around a C array. That is, it has a pointer to a contiguous data buffer of values. A Python list, on the other hand, has a pointer to a contiguous buffer of pointers, each of which points to a Python object which in turn has references to its data (in this case, integers).

Don’t Trust Sourceforge Downloads


In addition to injecting malware into their downloads (a practice they claim, hopefully truthfully, to have stopped), Sourceforge also presents an initial download page over HTTPS, then redirects the user to HTTP for the download itself, snatching defeat from the jaws of victory. This is fantastically irresponsible, especially for a site offering un-sandboxed binaries for download, especially in the era of Let’s Encrypt where getting a TLS certificate takes approximately thirty seconds and exactly zero dollars.

Previously: What Happened to SourceForge?

SwiftKey Keyboard Leaked User Information to Strangers

Cara McGoogan (via MacRumors, Hacker News):

A British keyboard app that uses artificial intelligence to predict the next word you want to write has suspended part of its service after users reported receiving predictions meant for other people, including email addresses and phone numbers.


“A few days ago, I received an email from a complete stranger asking if I had recently purchased and returned a particular model of mobile phone, adding that not one but two of my email addresses (one personal and one work address) were saved on the phone she had just bought as brand-new,” said the user.

The stranger with the new phone went through each letter of the alphabet and told the user the suggested words. They included names of the user’s friends and addresses for private servers they connect to for work.

Previously: iOS 8 Keyboards.

Update (2016-07-29): Rosyna Keller:

But congratulations to Microsoft’s SwiftKey for proving exactly why you should heed the full access warning!

Thursday, July 28, 2016

Apple User Interface Trends

Nick Babich:

For every action, there is a reaction. And in the world of digital design, flat design was a reaction against skeumorphism. Since anything on a screen will never truly look three dimensional, why not to stripping away illusory decoration and focus on functionality instead?


Recently, designers have begun to realize the usability issues of flat design and as a result, a more balanced interpretation of flat design has emerged. New solution sometimes referred to as ‘almost flat’ or ‘flat 2.0’ design. This design style is mostly flat, but makes use of subtle shadows, highlights, and layers to create some depth in the UI.

Nicholas W. Howard (via Rich Siegel and Gus Mueller):

In Apple’s view, an icon depicting a camera and a photo was too literal for an application that handles photos. Therefore, when iPhoto gave way to its replacement in 2015—Photos—the previous carefully-rendered icon gave way to this bland, meaningless rainbow abstraction.


Contrary to the ideas they express on this archival Mac Basics page (which, it must be observed, still features old icons), Apple nowadays prefers abstraction to metaphor, the less literal to the more literal, the two-dimensional glyph to the three-dimensional illustration. Allow me to explain how this ideological shift occurred in full-step with the design community at large.


Apple has gradually drained the color from the Finder sidebar, removing helpful visual distinctions from a part of OS X that millions of people interact with every day.

Nicholas W. Howard:

Now turn your attention to the barber pole’s successor, on the right. This redesigned loading signal, introduced in OS X Yosemite, reduces visibility in several significant ways: it measures less than half the height of its predecessor, uses only blue rather than blue-and-white, and makes its animated element—a slight glowing patch that moves across it—so bafflingly faint that even someone with superb vision can barely see it. The animation is the part that conveys “loading”: if Apple fails to make the “loading” state obvious, how will anyone ever know the loading is happening at all?


Clicking this “New Folder” button (if one can accurately call it a button), found in the Notes app, yields insignificant feedback: it almost feels as if nothing happens.


Hovering over Safari’s address bar no longer turns the cursor from the regular pointer to the “I-beam” text cursor; of course, one can still click on the address bar and type, but the cursor swap that whispered, “This is a place for typing,” is gone. In my testing, various search fields throughout the system sometimes replicated this problem, and sometimes not. OS X once handled this behavior properly and consistently. It no longer does.


It is a pity you did not know the “Detect Displays” button still exists. The button is invisible. You need only… hold down the Option key to summon it.


Some of these buttons in Pages have an arrow to indicate a dropdown menu. Others have no arrow but spawn a dropdown menu anyway. Apple has presented us with a guessing game.

John Gruber:

Luke Wroblewski posted an interesting side-by-side comparison of the Today view, Control Center, and standard sharing sheets in iOS 7 and the iOS 10 public beta. Much less transparency, more solid shapes in place of outlines, and more use of color.

End of Source-breaking Changes for Swift 3

Ted Kremenek:

The Swift team at Apple has reflected on this and decided what it “means” for Swift 3 to be source compatible with Swift 4 and later releases going forward. Our goal is to allow app developers to combine a mix of Swift modules (e.g., SwiftPM packages), where each module is known to compile with a specific version of the language (module A works with Swift 3, module B works with Swift 3.1, etc.), then combine those modules into a single binary. The key feature is that a module can be migrated from Swift 3 to 3.1 to 4 (and beyond) independently of its dependencies.

While the exact details of how we will accomplish this feat are still being discussed, here is a sketch of how this will likely work in the Swift 4 timeframe. The key enabler is a new compiler flag that indicates the language version to compile for (e.g., similar to the clang -std=c99 flag). The compiler flag will be provided by the build system you are using (e.g., Xcode, SwiftPM, etc.) on a per-module basis[…]


The great thing about this approach is that a single Swift 4 compiler is building all of the sources in an application. This allows us to roll out this approach before achieving full ABI stability — something that will be a goal for Swift 4, but is impractical to achieve for a Swift 3.x release. It also provides us a general framework in the future for handling source compatibility as Swift evolves.

See also: Swift Programming Language Evolution: Proposal Status (via Jacob Bandes-Storch).

Witch, Spaces, and Private API

Rob Griffiths:

Since that rejection, we’ve been looking for a solution to providing Spaces support for App Store Witch users … and unfortunately, we couldn’t find one: The only way we can get the window information we need from other Spaces is to use the private API. And that means that only the direct version of Witch supports Spaces, because we can use the private API. But App Store Witch no longer supports Spaces.


Gwendal Roué:

GRDB.swift is a Swift application toolkit that provides access to SQLite databases.


GRDB is protocol-oriented, and fetching methods are proper to the RowConvertible protocol.


This makes GRDB quite unlike other Swift ORMs that use class inheritance and make heavy use of objects mutability, like Realm and Core Data. Both of them provide uniquing and auto-updating records.

Update (2017-05-03): Gwendal Roué:

SQLite provides Data Change Notification Callbacks, and Compile-Time Authorization Callbacks. The first notify of modified rows, when the latter tell which tables and columns are involved in an SQL statement.

When you combine them, you are able to tell if an SQL statement has the opportunity to affect the results of another query. This is the recipe for database observation through RxGRDB (a reactive extension for GRDB over RxSwift), or FetchedRecordsController (a Swift version of Core Data’s NSFetchedResultsController).

Wednesday, July 27, 2016

LastPass URL Parsing Bug

Mathias Karlsson (Hacker News, Slashdot):

Stealing all your passwords by just visiting a webpage. Sounds too bad to be true? That’s what I thought too before I decided to check out the security of the LastPass browser extension.


I reported this to LastPass through their responsible disclosure page and the report was handled very professionally. The fix was pushed in less than a day(!), and they even awarded me with a bug bounty of $1,000.


Should we stop using password managers? No. They are still much better than the alternative (password reuse).

Although, taking a second to disable autofill functionality is a good move because this isn’t the first autofill bug we’ve seen, and I doubt it will be the last.


Disclosure: I work for AgileBits, makers of 1Password.

For browser extensions, the URL constructor would be even easier [for parsing]. (Yes, I know it says that IE doesn’t support it, but IE doesn’t have a proper extensions framework, so it’s irrelevant to this topic.)

Adobe Direct Download Links


The difference is that direct links to download the complete standalone/offline installers are not possible and no longer work if (only) the new approach is used. We at ProDesignTools have a long history of providing direct download links to all major Adobe software products, but now there is no way that we (or anybody) can make direct download links available for the new CC products released today (and beyond)!

[UPDATE (June 22nd @ 7pm) – We were just contacted by an Adobe Product Manager who says the company is aware of the situation and hoping to find a solution to restore direct download links and standalone installers for all tools in the new release! So we’re happy to report they are listening to customers and hearing your feedback. Please stay tuned to this page where we will keep you updated in the coming days!]

I had been using this page to get direct download links for Lightroom updates, but it hasn’t been updated in a while. I eventually found in a comment that the Lightroom 6.6.1/CC 2015.6.1 update is available here.

EFF DMCA Lawsuit

Matthew Green:

Today I filed a lawsuit against the U.S. government, to strike down Section 1201 of the Digital Millennium Copyright Act. This law violates my First Amendment right to gather information and speak about an urgent matter of public concern: computer security. I am asking a federal judge to strike down key parts of this law so they cannot be enforced against me or anyone else.


There’s a saying that no good deed goes unpunished. The person who said this should have been a security researcher. Instead of welcoming vulnerability reports, companies routinely threaten good-faith security researchers with civil action, or even criminal prosecution. Companies use the courts to silence researchers who have embarrassing things to say about their products, or who uncover too many of those products’ internal details. These attempts are all too often successful, in part because very few security researchers can afford a prolonged legal battle with well-funded corporate legal team.


In the United States, one of the most significant laws that blocks security researchers is Section 1201 of the Digital Millennium Copyright Act (DMCA). This 1998 copyright law instituted a raft of restrictions aimed at preventing the “circumvention of copyright protection systems.” Section 1201 provides both criminal and civil penalties for people who bypass technological measures protecting a copyrighted work. While that description might bring to mind the copy protection systems that protect a DVD or an iTunes song, the law has also been applied to prevent users from reverse-engineering software to figure out how it works. Such reverse-engineering is a necessary party of effective security research.

Removing Bit Flags in Swift Option Sets

Erica Sadun:

This code creates the complete .forbidAll set and then removes the local restriction.

var restrictions: AVAssetReferenceRestrictions = [ .forbidAll ]
restrictions .remove(.forbidLocalReferenceToLocal)

Interestingly, you can also pass .forbidAll without brackets in the current version of Swift and it will compile. […] I’m told that this option set syntax works because each element of an option set is itself an option set: [.forbidAll] is the same type and equal to .forbidAll. The array literal form of [.a, .b, .c] is syntactic niceness for creating an empty option set and then inserting (i.e. bitwise OR) each element)

Or you can write:

let restrictions: AVAssetReferenceRestrictions = .forbidAll.subtracting(.forbidLocalReferenceToLocal)

Tuesday, July 26, 2016

Verizon Acquires Yahoo

Marissa Mayer:

This sale is not only an important step in our plan to unlock shareholder value for Yahoo, it is also a great opportunity for Yahoo to build further distribution and accelerate our work in mobile, video, native advertising, and social. As one of the largest wireless and cable companies in the world, Verizon opens the door to extensive distribution opportunities. With more than 100 million wireless customers, a shared view of the importance of mobile and video ad tech, a deep content focus through AOL, Verizon brings clear synergies to the table. And with their aggressive aims to grow global audience to 2B users and $20B in revenue within the mobile-media business by 2020, Yahoo’s products and brand will be central to achieving these goals. Joining forces with AOL and Verizon will help us achieve tremendous scale on mobile. Imagine the distribution challenges we will solve, the scale we will achieve, the products we will build, and the advertisers we will reach now with Mavens – it’s incredibly compelling.


The addition of Yahoo to Verizon and AOL will create one of the largest portfolios of owned and partnered global brands with extensive distribution capabilities. Combined, AOL and Yahoo will have more than 25 brands in its portfolio for continued investment and growth. Yahoo’s key assets include market-leading premium content brands in major categories including finance, news and sports, as well as one of the most popular email services globally with approximately 225 million monthly active users. Additional technology assets in the advertising space include Brightroll, a programmatic demand-side platform; Flurry, an independent mobile apps analytics service; and Gemini, a native and search advertising solution.

Justin Fox:

Most of that value -- Yahoo’s current market cap is $36 billion -- is in the form of Alibaba shares, but even the $4.83 billion that the operating business is selling for is, for a media company in these Dark Ages for media companies, not all that bad. The biggest newspaper company, Gannett, has a market capitalization of $1.76 billion; the biggest magazine publisher, Time Inc., $1.68 billion; the biggest owner of local TV stations, Sinclair Broadcast Group, $2.86 billion. The media and entertainment conglomerates that own cable channels, TV and movie studios, theme parks and the like are still worth a lot more than that -- but Yahoo never possessed those kinds of assets.

I am probably being too fatalistic here. Perhaps, with a relentless, visionary leader in charge all along (Jeff Bezos, who founded Amazon a few months after Yang and Filo put together their original Web directory, springs to mind), Yahoo could have found a path to continued relevance. Even simpler, it could have just bought Google: Semel offered $3 billion in the summer of 2002, but refused to raise the price when Google’s founders turned that down. Still, reinvention is something companies usually fail at when their business models stop working. Yahoo at least succeeded in preserving some value in the midst of decline.

Nick Heer:

Back in February of 2008, Microsoft offered $44.6 billion to acquire Yahoo, in an attempt to better compete with Google. At the time, analysts speculated that it would be a difficult merger for antitrust reasons — an idea that seems positively quaint today. In the end, Yahoo rejected the offer[…]

Tom Harrington:

I wonder if Verizon realizes that Yahoo! Is an implicitly unwrapped optional. There might not be anything there when they try to use it.

Maciej Cegłowski:

If you didn’t want Verizon to have your private data, you shouldn’t have joined Flickr in 2004. Think ahead!

Daniel Jalkut:

Flickr would be a boon to any team independently fired up to take on Twitter, Facebook/Instagram, Snapchat, etc., but lacking resources.

Update (2016-07-29): jelenawoehr (via Hacker News):

To truly understand how soul-crushing it was to work at Yahoo in pre-Marissa times, you should have been in Sunnyvale (I was, on a business trip) on the day Marissa announced free food in the company’s cafeterias. People scrambled to stuff themselves as if the announcement would be taken back in a day or two. The coffee shops were stripped of pastries. Yahoos packed multiple boxes at the salad bar and hoarded them in break room refrigerators. You’d think that the announcement Marissa made was a coming price increase for lunches, not free food. Good news at Yahoo was treated as suspect and likely to change at any minute.

Pretty soon, it was obvious that the new CEO was listening — really listening.


And about devel-random, or “d-r”: I was pretty active on that. No one told Marissa about d-r, but just a couple of days after she got there, on a Saturday IIRC, she responded to someone on d-r. This sent shockwaves throughout the upper echelons, and soon senior management were clamoring to get on d-r. Most of them dropped out, exhausted by the volume; but she stuck around.

Also: she used to use pine(1) to read her emails. That increased her stature in my eyes, and those of quite a few other engineers.


The ads will begin appearing on the platform starting today. Tumblr remains one of the most popular blogging platforms, attracting over 550 million monthly users to its blogs. Tumblr creators will have an opportunity to share in the revenue from ads on their blogs. The company says that bloggers will have the ability to opt out of the program should they wish not to participate.

Update (2016-08-01): Jean-Louis Gassée:

Carriers’ fear of commoditization is alive and bad; how else to explain Verizon’s necrophiliac acquisitions of AOL and now Yahoo? For the acquired companies, it makes sense: If AOL had been a viable entity instead of walking its user base to the grave, if Yahoo’s revenues and profits had been on a growth curve, shareholders wouldn’t have clamored for an end to their suffering. For them, an assisted living home was their best choice.

But what about the acquirer? Does Verizon Sincerely Want To Get Richer? Will it try to increase its ARPU by piling up media properties, finding ways to decrease choices for its customers, somehow forcing them into paying for content bundles?

Eleventh Hour Swift 3 Reviews

Erica Sadun:

It’s crunch time. Several reviews are running concurrently for just a couple of days. All breaking changes must wrap up in the next week and Swift 4 design starts on 8/1.


In contrast, SE-0132 is huge. It really deserves more time, attention, and consideration than the Apple-imposed deadline will allow. Brent’s done a lot of work on perfecting this. If time permitted, it would have benefited from a few more weeks of discussion and revision.


Swift 3’s timeline concerns me. It will be ready for GM release along with Xcode 8 and iOS 10. I, personally, would prefer Swift 3 to be delayed six months or more to get it right the first time.

The Novelty of the App

Walt Mossberg (Hacker News):

I attacked my phone’s app landfill to learn how very many apps which once seemed interesting or necessary hadn’t made it into the toolkit of my life. How many had been superseded by better apps or by functions built into the phone or my other devices since their debut. How many were redundant or disappointing. Or, on the other hand, how many were great but did more than I ever needed.


But there are way too many bad, mediocre, or me-too apps. The good ones are too hard to find. And the novelty and joy of suddenly having access to millions of bits of interesting software has worn off for me and for many other people. While Apple now claims 2 million apps in the App Store, as long as two years ago analysts were reporting that most smartphone users didn’t download even a single app in an average month.

Update (2016-07-26): Oluseyi Sonaiya:

Yes, there are application categories that are resistant to this disintermediation, but they are likely far narrower than you may assume, and likely represent a tiny sliver of the average smartphone user’s installed apps.

Swift Type Aliases

Ash Furrow:

After deleting the typealias and replacing it with a class of the same name, we didn’t have to worry about changing function definitions and property types all throughout our codebase. We made the changes locally, in one file, and most all the rest of our code still compiled. Pretty cool! Here’s the relevant portion of the pull request that made that change.


Any time you use the same tuple type more than once, consider making a typealias. In this case, the code became a lot shorter and easier to skim and understand.


Objective-C developers, burdened with arcane syntax for blocks, use C’s typedef to isolate that syntax strangeness in one place. And even though Swift’s closure syntax is awesome, we can still benefit from Objective-C’s example – we can use type aliases for closure signatures.


I recommend using a descriptive typealias that is private to your file, and then extending that typealias so you can keep things neat and tidy. […] We’re still extending the view controller, but specifically we’re extending the typealias so that the extension has a helpful name. This is another way that typealias can help add semantic meaning to your code.

Too bad Swift extensions can’t be named directly, like Objective-C categories.

Monday, July 25, 2016

Microsoft’s Sweet Skype Solution

Richard Chirgwin (via AndrewMohawk):

In the same month Microsoft announced its alpha WebRTC-based Skype for Linux client, Redmond has put that native app and the native OS X Skype client on an end-of-life list.

This is because Skype is being rebuilt to replace its peer-to-peer architecture with cloud-centric code that supports Windows, iOS, Android and web browsers.

Microsoft has not given Mac and Linux users a date to dread. For now you can still get latest versions for all supported platforms – but it’s clear that full-fat Mac and Linux native clients aren’t something on which Microsoft wants to spend money forever.


The pace of change in our industry means that the devices and operating systems used by the majority of people shifts with time. Our commitment to deliver the best possible cross-platform experiences requires that we continually assess when it’s time to increase our focus on the platforms of the future. Sometimes this means that we must end support for some devices and operating systems.

Disable Find My Mac by Resetting NVRAM

Adam C. Engst:

There is one other problem that my friend Will Mayall alerted me to recently, which is that resetting NVRAM disables Find My Mac. Will discovered this on his own, but it turns out that others have run across the same fact over the past few years, as evidenced by a quick Google search. In essence, Apple stores the Find My Mac data in NVRAM, which is good for keeping it around even if the hard drive is removed, but bad in the sense that it’s easy to reset NVRAM — just restart while holding down Command-Option-P-R. A quick test confirmed the problem in OS X 10.11 El Capitan, and nothing has changed in the public beta of macOS 10.12 Sierra.

The only way to prevent Find My Mac from being disabled is to set a firmware password, which you must enter whenever you start up from a disk other than the usual startup disk. Plus, if you try to reset NVRAM, you’re prompted for the firmware password, and when you enter it, the Mac instead boots into Recovery mode. In fact, when you lock your Mac via Find My Mac, what it’s doing is setting a firmware password.

I still think Find My Mac is not worth the risks.

Swift Closure Capture Semantics

Olivier Halligon:

One important thing to note though is that in Swift the captured variables are evaluated at the closure execution’s time. We could say that it captures the reference (or pointer) to the variable.

For those of you who know Objective-C, you can notice that the Swift behavior is unlike Objective-C’s default block semantics but instead somewhat like if the variable had the __block modifier in Objective-C.


To capture the value of a variable at the point of the closure’s creation (instead of a reference to the variable itself), you can use the [localVar = varToCapture] capture list.

Reversing the WWDC Wall

Martin Conte Mac Donell:

I took ~50 (rather sloppy) photos of the wall and wrote a program to do image stitching using a cylindrical projection.


With the flat image and using OpenCV again I detected all the contours (letters) and created a (inverse) mask to remove the blue-ish background and only keep the letters.


This image was already good enough to run tesseract. In order to get the best results I trained tesseract with the San Francisco Mono font before running it.


Go find your app phrase here: you can also get the JSON representation of the wall from here.

Sunday, July 24, 2016

2016 MacBook – Two Months Later

Jeff Benjamin:

Although not everyone enjoys the low travel distance of the MacBook’s keys, it never posed a major issue for me. I have noticed, however, that some of the keys can get stuck as if infested with crumbs.


Jordan talked about the issues with USB-C in a prior post, and I can vouch for the fact that many of the USB-C peripherals that you’ll encounter can be finicky. I’ve experienced issues with devices abruptly disconnecting, devices that aren’t recognized without a reboot, and other annoyances.

Having a single USB-C port does occasionally present a problem, but that doesn’t bother me nearly as much as the wishy-washy interaction with some of my USB-C peripherals.

Previously: The 12-inch MacBook, Mistake One, The Developer’s MacBook.

Update (2016-07-24): Chris Turner:

I’ve had no keyboard issues, but fully concur on USB-C.

Sami Samhuri:

I bought the machine because I rarely use peripherals but have experienced USB disconnects with the battery I own.

But that is an exception. Since I don’t need max performance or use lots of peripherals, overall I love this MacBook.

Update (2016-07-25): Hampus Jakobsson (via Hacker News):

At the end of the day, I don’t think I am going back to my Mac. The ASUS [Chromebook] is super performant […] The biggest differences for me have been that there is never is a load time or memory running out.

Bridging Existentials & Generics in Swift 2

Benjamin Encz:

In an earlier blog post I pointed out some incompatibilities between type information that is statically known at compile time (Generics) and type information that is dynamically available at runtime (Existentials).


Given a heterogenous list of different instances that can be persisted we want to automatically find & call the DAO based on the type of object we encounter.


The .Self member, which would refer to the concrete type of the existential doesn’t exist in Swift 2. However, we can access the concrete type of the existential using Self from within protocols & protocol extensions.

Using a clever inversion of control we can use that Self type from within the PersistedType protocol (which all persisted types implement) to dynamically specify the generic type parameter of our GenericDAO<T>:

Exploring the App Store’s Top Grossing Chart

Graham Spencer:

One of the most striking things you’ll notice when browsing the Top 200 Grossing apps is that they are virtually all offered as free downloads. In my survey, just three apps were paid apps upfront; Minecraft (#33, $6.99), Grindr (#95, $0.99), and Facetune (#183, $3.99). The other 197 apps were free to download.


Whilst almost all the apps are free to download, it is also true that apps with In-App Purchases (IAPs) dominate the Top 200 Grossing charts. In my survey, just 2 apps did not offer any IAPs; Facetune (#183) and CBS (#200).


Games dominate the Top 200 Grossing charts, representing an overwhelming majority of 68% of the apps. The next closest is Social Networking at just 11% and comprised mainly of various dating apps.

Remote Code Execution With Image Files

CVE-2016-4631 (Hacker News):

An exploitable heap based buffer overflow exists in the handling of TIFF images on Apple OS X and iOS operating systems. A crafted TIFF document can lead to a heap based buffer overflow resulting in remote code execution. This vulnerability can be triggered via malicious web page, MMS message, iMessage or a file attachment delivered by other means when opened in applications using the Apple Image I/O API.


I was about to post that these exploits should be substantially mitigated by iOS sandboxing (you can get arbitrary code execution, but can’t get out of the exploited process’s sandbox without a second exploit), but then saw CVE-2016-4627 also in the 9.3.3 release notes, which is a local privilege escalation exploit that allows arbitrary code execution with kernel privileges.

It’s fixed in Mac OS X 10.11.6 and iOS 9.3.3.

Friday, July 22, 2016

Xcode 8 Illegal Hard Links Prevent Cloning

After I installed Xcode 8 Beta 3, I could no longer back up my hard drive. SuperDuper reported errors like:

Error creating hard link /Volumes/HD Clone 14A/Applications/ to /Volumes/HD Clone 14A/Applications/ for inode (null)

Cloning HD, error creating hard link for file in

At first I thought that the drive was damaged, but the error still occurred after reformatting it. SuperDuper’s Dave Nanian:

Structure became illegal (hard link into app bundle) in 10.10.3 - break link by duplicating file...

I assume they just did this in the latest beta. (It was previously an issue with Retrospect & Anaconda.)

Indeed, does contain a hard-linked file:

ls -l /Applications/\
total 48
-rw-r--r--@ 2 mjt  staff   6141 Jul 14 18:08 AuditIssues.strings
-rw-r--r--@ 1 mjt  staff     42 Jul 14 18:08 Localizable.strings
-rw-r--r--@ 1 mjt  staff  10726 Jul 14 18:08 LocalizableOSX.strings

Note the “2” for “AuditIssues.strings”. Here are the two files with the same inode:

find /Applications/ -name AuditIssues.strings -print -exec stat -f "%i" {} \;

The fix is to make the two files independent copies:

cd "/Applications/"
mv AuditIssues.strings AuditIssues.strings.old
cp AuditIssues.strings.old AuditIssues.strings
rm AuditIssues.strings.old

Now the files have different inodes:

find /Applications/ -name AuditIssues.strings -print -exec stat -f "%i" {} \;

Update (2016-07-22): Mike Bombich (developer of Carbon Copy Cloner) on the similar issue with Anaconda:

This turns out to be an issue specific to *.app/Contents/PkgInfo and *.app/Contents/Resources/*.lproj files. OS X does not want to permit the creation of a hard link between one of these items in an application bundle to another file in a non-application-bundle folder. I was unable to find an explanation for this behavior in Apple’s documentation, nor in the source code for HFS or the OS X kernel.

Pixellating or Blurring Text Creates Identifiable Patterns

Kashmir Hill (via Nick Heer):

If you’ve ever pixelated an email address or blurred a phone number before putting an image onto the internet in order to protect someone’s privacy, I’ve got bad news for you: Researchers at the University of California-San Diego have found that the popular Photoshop redaction techniques are decodable such that the underlying text can be read.

The researchers were able to recover text from a variety of redacted screenshots that they found online, said computer science professor Hovav Shacham by email. They were, for example, able to figure out the blurred email address in this screenshot of a conversation between a corrupt DEA agent and the then-CEO of Bitcoin exchange Mt. Gox.

What Exactly Is “Compressed Memory”?

John Siracusa:

In Mavericks, the OS has one more option before it has to resort to swapping: compressed memory. Mavericks will find the least-recently-used data in memory and compress it, usually to about half its original size. Et voilà, more free memory.


Memory compression is a triple play for Mavericks. It’s a performance win; compressing and decompressing data in RAM is much faster than reading from and writing to disk, even an SSD. It’s an energy win; the less time spent moving data between RAM and disk, the more time the system can spend in its idle state. And finally, it’s a capability win; Mavericks can handle much more demanding workloads than previous versions of OS X before crying uncle.

This seems like a great feature, but I’ve never fully understood how it’s reported in Activity Monitor. What do the “Compressed Memory” numbers for each process and the “Compressed” total actually mean? This AskDifferent post lists the columns in Activity Monitor, but commenter James K Polk has the same questions as me:

If the activity monitor says a given process uses 621.4 MB of memory and 615.4 MB of compressed memory, does that mean that the process is really only using up 6 MB of memory? Or that 615.4 out of 621.4 MB was compressed down to some unspecified size? Or something else?

In other words, does a high number for Compressed mean that the compression has been effective or that the system is nearing capacity because most of what can be compressed already is?

It’s also not clear how compressed memory interacts with the other reported numbers. For example, why did Siracusa’s App Memory go down when the Compressed memory went up? I would have expected that App Memory would include the part that’s compressed.

And, presumably the memory that is paged out to disk is compressed. Does Swap Used take that into account?

Update (2016-10-03): Activity Monitor in macOS 10.12 has a tooltip for the “Compressed Memory” column, which says “Memory sent to the VM compression.”

Sandboxing Wisdom

Daniel Jalkut (tweet):

[I’ve] managed to produce two versions of my app, one of which causes the sandbox container to be apparently unwritable to the other after running! Specifically, preferences are not saved and console messages indicate an attempt to write preferences outside the host app’s sandbox.


These kinds of issues scare the bejeezus out of me because I really fret my users running into data migration problems after I ship an update, and because the relative opacity of the sandboxing system makes a lot of issues very hard to debug.

Daniel Jalkut:

In a nutshell: for the past 4 years or so, sandboxing has been a massive, amorphous “bug” that I have to wrap my head around. So, so tired.

Rarely a day goes by when I don’t worry that I made a huge mistake betting on sandboxing and MAS for the long run.

Peter Maurer:

Answer from a very tired sandbox wrangler: Don’t do it. Don’t waste your time on custom settings, etc. Instead, import once…

…via open dialog automatically, then make additional imports (for whatever reason) available via menu or preferences.

Ilja A. Iwas:

We released GarageSale 7 this week, still sandboxed, but won’t be submitted to the MAS. Ah, that sweet feeling of relief.

Daniel Jalkut:

This tool [asctl] appears to offer extensive insight into the sandbox’s understanding of containers. I wish I had discovered it earlier!

Apple doesn’t seem to have posted the asctl man page, but this online version has the same date stamp as the man page on my 10.11 system.

Thursday, July 21, 2016

“This Regular Expression Has Been Replaced With a Substring Function”

Stack Exchange (Hacker News):

The direct cause was a malformed post that caused one of our regular expressions to consume high CPU on our web servers. The post was in the homepage list, and that caused the expensive regular expression to be called on each home page view. This caused the home page to stop responding fast enough. Since the home page is what our load balancer uses for the health check, the entire site became unavailable since the load balancer took the servers out of rotation.


The regular expression was: ^[\s\u200c]+|[\s\u200c]+$ Which is intended to trim unicode space from start and end of a line. A simplified version of the Regex that exposes the same issue would be \s+$ which to a human looks easy (“all the spaces at the end of the string”), but which means quite some work for a simple backtracking Regex engine.


This is not classic catastrophic backtracking (talk on backtracking) (performance is O(n²), not exponential, in length), but it was enough. This regular expression has been replaced with a substring function.

Tuesday, July 19, 2016

AppleScriptObjC in Script Debugger 6

Shane Stanley:

The most important new AppleScriptObjC feature, and the most obvious, is how Script Debugger displays results. Instead of «class ocid»..., you will see much more.


First, although case matters in AppleScriptObjC, code completion is case-insensitive. You can type NSS or nss, and get the same result; the correct case will be inserted. You can also use the tab key within the completion list; this is very helpful where there are multiple methods with similar names.


Script Debugger includes terminology for the following frameworks: Foundation, AppKit, Quartz, AVFoundation, AddressBook, Contacts, CoreImage, CoreLocation, CoreWLAN, EventKit, JavaScriptCore, MapKit, OSAKit, and WebKit. The information is collated from the frameworks themselves and from header files, with some filtering applied to remove many methods that cannot be used from AppleScriptObjC.


Finally, there is an important limitation. Script Debugger runs scripts on a background thread, so any code that needs to be run on the main thread needs to use the method performSelectorOnMainThread:withObject:waitUntilDone: to do it. If not, you are likely to freeze or crash Script Debugger.

Previously: Script Debugger 6.


Scott Perry:

PSA: SQLite on Sierra/iOS 10 is built with SQLITE_ENABLE_SQLLOG, which makes it easy to create replayable SQL logs


This file contains experimental code used to record data from live SQLite applications that may be useful for offline analysis. Specifically, this module can be used to capture the following information:

  1. The initial contents of all database files opened by the application, and
  2. All SQL statements executed by the application.

The captured information can then be used to run (for example) performance analysis looking for slow queries or to look for optimization opportunities in either the application or in SQLite itself.


At runtime, logging is enabled by setting environment variable SQLITE_SQLLOG_DIR to the name of a directory in which to store logged data.

MacKeeper Threatens YouTube Video Maker

Mike Wuerthele:

Infamous software developer MacKeeper has demanded that four videos critical of its maligned tune-up utility suite be removed from the internet, threatening the teenager behind the videos with $60,000 in court costs and legal fees.


In August 2015, ZeoBIT, creators of MacKeeper, agreed to a settlement in 2015 to put $2 million in a compensation fund to cover attorney fees, refunds and administrative costs to U.S class action claimants who purchased MacKeeper prior to July 8, 2015.

The complaint alleged that “ZeoBIT intentionally designed MacKeeper to invariably and ominously report that a user’s Mac needs repair, and is at-risk due to harmful (but fabricated) errors, privacy threats, and other computer problems, regardless of the computer’s actual condition.” Court documents state that 513,330 people are registered owners of the software in the U.S.

Previously: MacKeeper.

Apple Music Learns From iTunes Match

Kirk McElhearn:

When Apple Music was released just over a year ago, Apple also debuted iCloud Music Library, a way of storing your iTunes library in the cloud. There were two ways to seed the cloud, either with iTunes Match or Apple Music. If you were an iTunes Match subscriber, matching your songs in your local library to your cloud library was done one way, and if you were just an Apple Music subscriber, matching was done differently.

This created some confusion about the way tracks were matched and stored in iCloud Music Library. Now, Apple is changing this, and will use the same matching method for both services. The company said that Apple Music now uses acoustic fingerprinting and provides matched files without digital rights management (DRM), or copy protection, just like iTunes Match.

Before, if you only had Apple Music, you only got the problematic metadata-based matching.

John Gruber:

I’m sure there are reasons for the way things are, but from the outside, combining iTunes Match and Apple Music should have been there from day one.

See also: Jim Dalrymple, iMore.

Monday, July 18, 2016

The Strange Case of the System Preferences Window Width

Tim Schröder:

Apparently, the System Preferences window on my computer running OS X El Capitan was considerably wider than 668 (or even 595) pixels. Some research later it turned out that the width of the System Preferences window depends on the system language: The window will be 668 pixels wide when English is the primary language, but will have a different width for other languages. For example, with German set as primary language, as it is on my computer, the window is 762 pixels wide.


While Apple’s preference panes that come with OS X dynamically adjust their width to the actual width of the System Preferences window, this is not true for any custom preference pane, that will instead be displayed centered with more or less wide blank margins.

Medium URL Fragment Tracking

sime_vidas (via Ole Begemann):

Notice how this URL does not contain a hash (at the end). If you visit this URL, a hash will be added to the URL after the page loads. The hash value is some jumbled combination of letters and digits. What’s the purpose of this hash?


The idea is that when you share that page you will share it with the hash at the end. The visitors of the link you share will have their own hash generated to track their referrals. Medium will be able to see who visited the link that you shared, and build a tree of referrals (who the users you referred shared it with, who they shared it with etc). It’s a great metric to use in analytics when looking at user acquisition through social channels.


I hate this, I can’t tell you how many times I get a duplicate bookmark because I thought I didn’t have the article already in my bookmarks.

Exponential Time Complexity in the Swift Type Checker

Matt Gallagher:

But the line doesn’t get past the Swift type checker. Instead, it emits an error that the expression is too complex to solve. It doesn’t look complex, does it? It’s 5 integer literals, 4 addition operators, two negation operators and a binding to a Double type.

How can an expression containing just 12 entities be “too complex”?


If you don’t typically combine these features in your code, then you’re unlikely to see the “expression was too complex” error. However, if you are using these features, it isn’t always straightforward to suddenly stop. Mathematics code, large “function”-style expressions and declarative code are easier to write with these features and often require a complete rethink to avoid them.


A note about this approach though: unlike other languages, Double(x) is not equivalent to x as Double in Swift. The constructor works more like another function and since it has multiple overloads on its parameter, it actually introduces another overloaded function into the search space (albeit at a different location in the expression).


Posts like this: “[swift-dev] A type-checking performance case study”, indicate that the Swift developers believe resolving function overloads in the type checker is inherently exponential. Rather than redesigning the type checker to eliminate exponential complexity, they are redesigning the standard library to try and skirt around the issue.

Joe Groff:

Changing stdlib interfaces is done with an eye toward fixes like SE-0091 that mitigate inherently exponential work.

Most of the proposals around function labels and types are trying to kill type checker tech debt too.

“Rewrite the type checker” is up there on list of things we want to do.

Previously: Speeding Up Slow Swift Build Times, Swift 1.0 Performance and Compilation Times, Slow Swift Array Type Inference, Swift Type-checking Performance Case Study.

The Secret Life of Types in Swift

Slava Pestov:

I’m going to attempt to start by giving an overview of how types work in Swift, from the parser down to the lower layers of code generation in the frontend. Swift is a strong, statically-typed language with an advanced type system more reminisicent of functional languages such as OCaml and Haskell than something like C, so this seems like as good a place to start as any.


Types in Swift form a mini-language in of themselves, with a grammar consisting of nominal types as leaves, and structural types such as function types as interior nodes. Types are formed from TypeLocs and TypeReprs early on in semantic analysis. Further down in the compiler, sugar is removed and types are canonicalized, simplifying structural walks and equality comparisons. Substitution is a fundamental operation frequently used in the implementation of generics, and it is important to think about the role of types and declarations when performing substitutions for member access. Various higher-order operations simplify tedious boilerplate when manipulating types throughout the compiler.

Slava Pestov:

Now, let’s peel back a layer and dive into the type system of SIL, the Swift Intermediate Language. SIL adds a layer of detail missing from formal types, drawing a distinction between values and addresses, and making function types more explicit by introducing explicit annotations for argument and return value conventions.


At this point, we still cannot compile our code, but at least we can detect a type mismatch at the level of SILFunctionTypes, instead of just mis-compiling incorrect code. A situation where the formal types of the expressions match, but the lowered types do not is called an “abstraction difference”. Abstraction differences are handled by SILGen wrapping the substituted function value inside a re-abstraction thunk.

The re-abstraction thunk forwards arguments, calls the function, and forwards the result, taking care to handle any abstraction differences in the arguments and results. If the substituted argument is trivial but the original argument is passed indirectly, the thunk will load the value from its address and pass it to the substituted function. Similarly, if the substituted result is trivial but the original result is returned indirectly, the thunk takes the substituted result value, and stores it into the indirect return address given to the thunk.

Sunday, July 17, 2016

Swift Classes to Be Non Publicly Subclassable by Default

SE-0117 (original, revision 1, revision 2):

The major observation here is that not all classes make sense to subclass, and it takes real thought and design work to make a class subclassable well. As such, being able to subclass a public class should be an additional “promise” beyond the class just being marked public. For example, one must consider the extension points that can be meaningfully overriden, and document the class invariants that need to be kept by any subclasses.

Beyond high level application and library design issues, the Swift 1 approach is also problematic for performance. It is commonly the case that many properties of a class are marked public, but doing this means that the compiler has to generate dynamic dispatch code for each property access. This is an unnecessary performance loss in the case when the property was never intended to be overridable, because accesses within the module cannot be devirtualized.

Chris Lattner:

As expected, this proposal was extremely polarizing, with valid arguments on both sides. The opinions held by supporters and opposers are held very strongly, and hundreds of emails were generated in a healthy debate about this topic.


On the first point, there are three related arguments against SE-0117:

First is that clients of Apple frameworks often choose to subclass classes that Apple publicly documents as being “not for subclassing”, as a way of “getting their job done,” typically as a way to work around claimed bugs in Apple frameworks. The core team and others at Apple feel that this argument is analogous to the argument that Swift should “support method swizzling by default”. Swift loves dynamic features, but has already taken a stance against unanticipated method swizzling, by requiring an API author to indicate where they allow method swizzling with the ‘dynamic’ keyword.

Yes, it’s absolutely analogous. It’s become obvious that “unanticipated” dynamism is not something that Apple wants to support with Swift. It’s clearly not Swifty, if by Swifty you mean the vision of the core team.

Second is that clients of some other public API vended by a non-Apple framework (e.g. a SwiftPM package) may end up in a situation where the framework author didn’t consider subclass-ability, but the client desires it. In this situation, the core team feels that a bigger problem happened: the vendor of the framework did not completely consider the use cases of the framework. This might have happened due to the framework not using sufficient black box unit testing, a failure of the imagination of the designer in terms of use cases, or because they have a bug in their framework that needs unanticipated subclass-ability in order to “get a job done”. Similar to the first point, the core team feels that the language is not the right place to solve this problem. Instead, there is a simple and general solution: communicate with the framework author and get them to add the capabilities that you desire.

Yes, there’s a bigger problem, and, yes, the language can’t “solve” this problem. That doesn’t imply that the language should forbid building a temporary workaround. In my view, the language should be helping to get the job done, not enforcing an idealized solution that may not even be possible because the vendor may be busy, uncooperative, or no longer available.

Third is a longer-term meta-concern, wherein a few people are concerned that future pure-Swift APIs will not consider subclass-ability in their design and will accidentally choose-by-omission to prevent subclass-ability on a future pure-Swift API (vended by Apple or otherwise). The core team feels that this is an extremely unlikely situation for several reasons. First of which is that it heavily overlaps the first two concerns. More significantly, any newly-designed and from-scratch APIs that are intended for Swift-only clients will make use of a breadth of abstractions supported by Swift—structs, enums, protocols, classes.

Indeed, this proposal is just one piece of a large trend. Swift is chipping away at dynamism for classes: subclassing, swizzling, hiding private classes and private methods. And the future, it appears, is APIs that rely more on structs, enums, and protocols, which are even more static. There are definitely some benefits to this direction, but I am concerned that it will lower app quality over the long term. This specific proposal may not have a huge impact because it affects neither Objective-C APIs (past) nor non-class Swift APIs (future).

To reiterate, as a summary, the core team agrees with conviction that it is the right default for public classes to be non-subclassable outside their module, unless they carry some additional indication by the API author that the class was designed to be subclassed.

Here’s my slightly exaggerated summary of the two points of view:

Me: I agree that dynamism shouldn’t be abused, but there are library bugs and limitations, and sometimes we need these tools to cope with them. I hate coding these workarounds and would love to remove them and unclutter my code as soon as the underlying issues are resolved. But bugs that affect the customer are much worse than temporary ugly code. Of course, ideally library vendors wouldn’t ship any bugs, but this is the real world. Nudging the API author to “think carefully” is well-intentioned but naive. The proposed solution is worse than the problem.

Apple: Making library vendors opt into dynamism will encourage them to think more about their designs. They will foresee how their APIs will be used. Making it impossible for developers to work around library bugs will foster communication with the vendor, who will then fix them. The fixing will be easier because the vendor won’t have to take into account developers’ workarounds. We don’t particularly care if your app is broken for months or years while waiting for a bug to be fixed. Developers won’t feel pressured to work around library bugs because competing apps will be subject to the same restrictions.

Here are some of the more interesting comments from the discussion:

Kevin Lundberg:

I do not want to be constrained by authors of libraries or frameworks into interacting with a system in only the ways they forsee. By making the default be non-subclassable, if a designer does not put thought into all the ways a class can be used then I as a consumer of the library am penalized.

Another point I can think of is mocking in tests. I’m unaware of any fully featured mocking solution in swift, and subclassing a class to override methods for mocking purposes is one way to make unit tests work with third party dependencies. If a class was not designed for subclassing, then writing tests where the class’s behavior needs to be suppressed would not be possible.


I’ve had issues before when working with third party C# code where the author didn’t declare a method as virtual, but the problem I was trying to solve required me to override that method. Luckily the code was open source, so I just embedded and edited the source into my project instead of using the prebuilt binaries, but that becomes a maintenance headache, and it is not possible for components that do not make the source available.

L. Mihalkovic:

Can’t really help for feel like it is training wheels all around… or padlocks on every kitchen cupboards.

Kevin Lundberg:

It’s not possible to accidentally override a method like it is in java or objective-c, so the proposal won’t help people who might accidentally override something. If a developer tries to override a method or class they probably have a specific reason in mind for doing so, and taking away the options that this ability presents, by nature of it being the default behavior, doesn’t sit well with me as it has the potential to take away a good tool from my toolbox.

I agree that API authors should have the power to restrict this dimension of a type’s usage, but I feel that it should be a conscious choice to do so.


I really enjoy designing elegant, well-defined APIs and this behaviour for classes has bothered me for some time.

On the other hand, I have also subclassed things that weren’t meant to be subclassed (e.g. I believe UITabBar was an awkward one back in the day, and UINavigationController has so many broken behaviours it’s almost a requirement to subclass and patch it, even though I believe Apple disapproves). I could fall back to Objective-C, but that’s an implementation detail. In some theoretical future with an all-Swift UIKit, what do I do about those issues?

After thinking about it, I decided that in this theoretical future, no App would be able to use those hacks, and so all Apps would have the same broken behaviours and degrade the quality of the platform to such an extent that Apple is forced to do something about it. Ultimately, that’s exactly what we want; they need to see broken Apps everywhere in order to prioritise fixes. That improves code quality all-around.

Tino Heth:

I can’t fight the feeling that many fans of ideas like this believe that good designed libraries come for free by simply adding restrictions — which, at least in my opinion, just isn’t true: No matter what the defaults are, good libraries are hard to build, so I predict this proposal would not only fail in increasing framework quality, but also will make it much harder for users of those frameworks to work around their flaws, which are just a natural part of every software.

Michael Peternell:

I think it’s a step backwards. […] It’s not bad to allow sealing of classes. I don’t see real value in it though. And “sealed” should not be the default. Either the class if final or not: for me that’s a valuable distinction. “sealed vs. unsealed” is not.

Paul Cantrell:

In all the OO languages I’ve used, classes are open by default. That raises a question: is this really a good idea? Aren’t we ignoring established precedent? What about all the unauthorized subclassing we’ve done in the past?

I’m sympathetic to those accustomed to Objective-C, Ruby, Javascript, Python, and other languages with more fungible type systems who argue that it’s unreasonable to expect us all to use classes only as library authors intend. Monkey patching has saved many a day, they’d say. And it’s true, it has! I rely on it.

My counterargument: the ecosystem is changing.


The big exception to this is Apple itself, and the direction of this proposal implies a large cultural shift in how Apple deals with its developer community. Having the Swift language so aggressively prevent the dubious, brittle, bad-idea-but-it-gets-the-job-done workarounds that the Obj-C runtime allowed means that Apple’s platforms are going to have to be more consciously extensible, more transparent in their design decisions, and more responsive to unanticipated needs.

Tino Heth:

With “sealed by default”, the situation changes: Users are protected from some simple bugs, but nonetheless, they’ll encounter situations where the library doesn’t do exactly what they want. So, you take a look at the source, find the problem, and fix it. It’s no bug at all, it’s just a tiny degree of freedom that is missing because it wasn’t important to the author. You can create a pull request as well, but it doesn’t offer a real improvement to the author, who is already busy with dozens of similar requests by other users -> you end up with a custom branch with all the housekeeping associated with it.

So overall, I’m quite sure that the proposal won’t improve software quality, but rather degrade it.

Ricardo Parada:

I am afraid that developers will not take the time to specify which methods are overridable resulting in libraries that are difficult to patch, extend.

In my 26+ years of object-oriented design and programming (other languages, Objective-C since 1990 and Java since 2001) I have worked with object oriented libraries and subclassed methods that the authors probably never anticipated. I have been able to fix problems, enhance classes by creating subclasses with fixes and enhanced behavior.

In java for example I have seen that sometimes I would have been able to fix bugs or enhance the existing classes had the author not chosen a method to be protected or private. Sometimes they had a good reason but sometimes they didn’t. Is have been able to survive using an awesome library that was discontinued and end-of-lifed thanks to subclassing that has allowed me to fix problems and enhance over the years as the Java language kept evolving.

Jakub Suder:

I agree that subclassing library classes that were not explicitly planned to be subclassed might sometimes lead to bugs. But just because a technique or feature of a language allows you to sometimes make mistakes, it doesn’t mean it should be completely disallowed. By this logic, we’d have to remove all instances of "!" any anything related to pointers from Swift. It’s possible for a feature to allow you to shoot yourself in the foot, and still be useful at the same time.


I don’t think the problem is significant, and I’m afraid the proposed solution is worse than the problem itself.

Jonathan Hull:

I don’t think this proposal will achieve its stated objective of forcing people to think about subclassing more. It will just add confusing boilerplate.

Things like Swift optionals work well because they make the (often forgotten) choices explicit in the context that they are used. In the world of Human Factors, we call it a forcing function. This proposal has the inverse structure, and will be ineffective, because the “forcing” part of it shows up in a different context (i.e. trying to use a framework) than the decision is being made in (writing the framework). This type of thinking leads to things like Java and the DMV.


Those who were prone to be thoughtful about their design would have been anyway. Those who are not thoughtful about their design will just leave these annotations off… leaving us with no recourse to extend/modify classes. When people complain, they will add the annotations without actually thinking about the meaning (i.e. stack overflow / the fixit tells me I need to add this word to make the compiler happy). All this does is put framework users at the mercy of the framework writers.

Brad Hilton:

I […] give a strong, strong, strong -1 to this proposal. To make classes non-subclassable by default is only going to lead to unanticipated pain and frustration. Also agree with other comments that subclassable and overridable conflate access control with class behavior. If we want to make it possible to define a class as non-subclassable to external users, I’d agree to something more consistent with existing swift access control like public(final) as has been proposed by other commenters. However, I agree that making classes final by default is a bad idea that will create a larger problem that it solves.

Jean-Daniel Dupas:

I can’t count the number of times it save my hours [to be] able to override arbitrary classes and methods.

Sometimes to simply add log point to understand how the API work. Other times to workaround bugs in the library. Or even to extends the library in a way that the author did not intent in the first place, but that was perfectly supported anyway.

I already see how libraries author will react to that new default. They will either don’t care and mark all classes as subclassable, or find to burdensome to get subclassability right and prohibit subclassing all classes.

Jean-Daniel Dupas:

Nonetheless, being able to subclass any class allowed me to solve a bunch of very bad memory leaks in the new NSCollectionView implementation, and made it usable for my project.

Thinking than you will have to work only with perfectly design libraries without any bug is utopian. And thinking you can predict every usages of your library by users is short sighted IMHO.

John McCall:

I sympathize with the argument about wanting to fix bugs and add features via override, but that’s never been maintainable in the long term; you always just end up with superclasses that everyone is terrified to touch because every subclass has its own invasive “fixes”, and that’s even when working within a single codebase. With libraries, you can pretty quickly get locked in to a specific version because your customizations don’t work with new releases; either that, or the maintainer just decides that they can’t fix of their mistakes and so goes off to rewrite it from scratch. Either way, it’s not good for the ecosystem.

Jeremy Pereira:

Thirdly, this is a huge breaking change. Not only is it a breaking change, but for a lot of people, the breakages will be outside of their control. Consider if I publish a module with a class with public methods and you subclass it in your code. Once this change is implemented, my code will still compile and pass its unit tests but your code is now broken and you are dependent on me changing my code to fix your code.

Tino Heth:

Defaults matter, because they transmit a message:

Every rule and obstacle we add to Swift is a statement that says “we favor bureaucracy over freedom”, and this will affect the community evolving around the language.

When you use a library in a way that wasn’t anticipated by its author, you’ll ran into issues occasionally; nonetheless, I think we should struggle for an open ecosystem that encourages others to experiment and fail, rather than to limit their possibilities in a futile attempt to protect them.

Garth Snyder:

I’ll just note that this proposal reminds me of the fortune file entry, “Whenever you see a sign that says ‘no exit’, it means there is an exit there.”

Specifically, under this proposal, whenever you see a nonsubclassable, non-final class (which I suppose would in fact be any class not explicitly marked as subclassable), it would mean that the library implementor IS in fact subclassing it. Otherwise, it would just be final. Therefore, it’s by definition a perfectly reasonable class to specialize. The implementor just doesn’t want YOU doing it.

Maybe the implementor has a very good reason for this. But in the modal case, probably not.


There should absolutely be a way to say “don’t subclass this.” But do you really need anything more than a comment that says “don’t subclass this?” Perhaps as a compromise, and as a concession to the value of automated checking, this feature could be reformulated as a warning system. A class marked as nonsubclassable could be extended only be including a corresponding acknowledgment keyword in the extending class, a kind of Swift version of -IHaveBeenWarnedThatAPFSIsPreReleaseAndThatIMayLoseData.

Paul Norton:

Because of that experience, I am fully convinced that library authors will never imagine all the ways developers will use a library’s API.


I also think it would eventually force a second change; some sort of override to the default, at which point we’ve extended the language further to get closer to where we started.


Personally, Im not against sealed by default, but I think there are cases where closed source libraries have certain cases where workarounds are necessary, and just sealing by default will prevent those cases.

One could say, “well just use an open source one, or change vendors” but its not that easy in The Real World™ where we get certain SDKs shoved down our throats by the suits… and while that may be a separate issue to the one at hand, its still a problem that won’t resolve itself by simply locking down things…

In my own case, Ive fought with NSBrowser / NSTreeController in the past and the only way to resolve things was to subclass (and no, waiting 1 or 2 years for a fix is not acceptable if you already have a product in the wild).

Jonathan Hull:

Please stop saying that this proposal will bring more consideration to the design of libraries. It isn’t true. I haven’t even seen an argument for why it would be true, it is just taken for granted that it is true.


It sounds good, but at the end of the day, people are human and they will make mistakes. Best to either catch those mistakes in the context where they happen or to mitigate the effect of it. This proposal basically forces you to feel the full effect of other people’s mistakes (thinking that it will discourage them from making them in the first place).


The current proposal will only cause massive problems down the line, IMHO. We will find an escape hatch is needed, but we will have made optimizations based on assumptions of finality which prevent us from easily adding one.

L. Mihalkovic:

There are IMO no advantages to sealing by default. If there were, I cannot imagine how they can have so consistently eluded the designers and maintainers of so many great OOD languages in the last 30 years. Does it mean it is just a matter of time for the core team to take it the c++ standardization committee to make sure C++ gets enhanced the same way?

L. Mihalkovic:

After 30 years of Objc’s msg dispatching being touted as not a performance problem, all we hear for the last couple WWDC (did you pay attention to the ‘they are getting it’ during one of the session) is how much every single dynamic dispatch must be chased out of our systems, culminating with this ‘seal by default is best for you’. Why can’t developers [writing] code be made responsible for writing well or not?

Jonathan Hull:

There is also a dangerous difference between helping the programmer catch mistakes (e.g. don’t accidentally subclass the wrong method) and trying to prevent them from coding in a style you disagree with. I have been seeing far to many proposals of the second variety of late.

let var go:

It is the opposite of nearly every other OOP language’s behavior and the opposite of what anyone will expect who is coming to Swift after doing any OOP programming anywhere else. While I believe that Swift should be free to buck the trends of other languages where there are significant advantages, changing something this fundamental will introduce a new learning curve for even experienced programmers. The advantages better be significant, otherwise you are introducing a frustrating quirk into the language without any great benefit in exchange. But the advantages of default non-subclassability are marginal at best.


The developers who don’t make careful design decisions will just go with the default - in this case they will just leave the default in place, and their public classes will not be subclassable. That doesn’t improve anything, it just makes sloppy, poorly written code harder to fix in the off-chance that you are stuck working with it. In other words, quality will not improve, but productivity will suffer, because it will be harder to develop workarounds for the problems that will inevitably appear in even the best-designed APIs.


But this whole issue of default non-subclassability doesn’t fall into that category of “safety” at all. It doesn’t help a programmer produce safer code. The same hidden flaws will persist whether non-subclassability is the default or not. The difference is that those same hidden flaws will be more difficult to deal with after the fact.

Colin Cornaby:

The “final should be default because adding final after the fact is destructive” arguments are interesting, but ultimately not convincing to me. If you thought your class was safe for subclassing, but it ultimately wasn’t, this solves nothing. You’ll mark your class as subclassable, and you will ship it, and there will be issues, but it will still be too late to take things back. If you know your class is not safe for subclassing, you should mark it as final. There is no advantage here in that scenario.

This is all part of building a safe public API. Public API design can be difficult, but if you don’t understand safe subclassing designs, you are likely to miss the issues and mark your class as subclassable and ship it anyway. Again, the best way to tackle this is to find better ways to surface the issues. Making final the default still doesn’t solve the core issues of people not understanding the right design.

Jonathan Hull:

My point is that you are completely ignoring an entire class of risk that has a real-world $$$ cost. Every time I have to use a framework under this proposal, I am now completely at the mercy of the author. In the case of open source frameworks I can at least make a fork, but for closed source frameworks (often from large companies where us using their framework has been negotiated by the bosses) you have now just added weeks to my development cycle while I wait for big-company-who-doesn’t-really-care-that-much to update their stuff.


Are you offering to explain to my boss/client why I can’t add the feature in a reasonable timeframe like I can with Objective C frameworks? That it may not even be possible now in Swift even though the Android guy just did it in a few hours?

Do you know what I am going to tell my boss/client? “Don’t use Swift for frameworks” and “Try to avoid partnering with companies that have Swift frameworks”. “It is too much of a risk”. “We are giving them too much control over the future of our product…” I mean, it even affects the prices that companies can charge for crappy frameworks. If I am asking for a bunch of features that I need them to add to provide a basic feature, that affects negotiations/price (vs a world where I can add it myself if needed). Sealed-by-default gives them leverage.

Tony Allevato:

The argument shouldn’t be “we need open subclassing or we can’t fix broken APIs”; that’s a false choice. It should be “we need something to fix broken APIs”, and that “something” should be proposed and the need for it should be argued in its own right.

David Sweeris:

-1 for this proposal, but +1 for solving the issues it raises

Regardless of what ends up being the defaults, I’m a very strong -1 on conflating visibility and subclassability/extendability.

Jon Akhtar:

I completely agree, the programmer should have as much power as we can give them, even if it means allowing them to shoot themselves in the foot, because at the end of the day Swift isn’t an academic exercise, it is a real world programming language, so it should be optimized for solving real world problems not having some kind of technical, philosophical, and stylistic perfection when those come at the cost of the former.

Update (2016-07-17): Daniel Jalkut:

It’s absurd to imagine you can predict all valid subclassing patterns. […] Apple, nor any framework provider, can provide functionality sufficient to cover the breadth of innovation. Clever hacks built the industry.

To be clear, although the proposal was “returned for revision,” it is essentially accepted. It’s just the syntax that’s being debated now (e.g. changing subclassable to open).

Update (2016-07-18): Friedrich Markgraf:

Same issue with Apple’s removal of Xcode plug-ins it prevents unanticipated innovation

Marco Scheurer:

Another nail in the coffin. Swift: “No programmers except myself should be trusted”

Daniel Broad:

If objective-c had been like this we would have had only default coloured navigation bars until ios4.

Casey Liss:

I’m… not keen on the idea of final by default.

Do not want.

Ilja A. Iwas:

This is why you don’t let compiler engineers control the fate of your computing platform(s)

Orta Therox:

This feels like a real step against getting things done in Swift

Nick Lockwood:

I’m in two minds. It won’t impact existing Cocoa APIs as they’re all ObjC, & future Swifty APIs will use fewer classes anyway.

Radek Pietruszewski:

But values are never the stuff in need of hacking, it’s the classes that do complex stuff.

Mike Ash:

This “claimed bugs” language makes me uncomfortable.

Adam R. Maxwell:

I’ve been worried that guys designing languages/frameworks don’t maintain GUI apps using them, or they’d know there are bugs.

Florent Pillet:

I’ve been fighting with lack of dynamism on Android. When you find issues in the OS you have to rewrite the whole class

Paul Haddad:

My outsider’s perspective of Swift: X can be misused, so we’re banning X.

Seems like coding with safety scissors.

Robert Cottrell:

It seems like a lot of recent decisions are justified “it’s no so bad because you can just go and ask the vendor to fix it”.

Leon Breedt:

yes Radar is a fantastic bidirectional communication tool with a responsive vendor, what could go wrong?

Nacho Soto:

if Apple knows there’s no workaround for their bugs, maybe they’ll get better at shipping quality software.

Max Seelemann:

Well, apparently the core team never had to write software against Apple frameworks. They have no idea how buggy they are.

Friedrich Markgraf:

They just bring their stuff over to Craig’s house if it doesn’t work. They have a mechanism for fixes; we don’t.

Jeremy Tregunna:

This will raise the cost of developing software, and make bugs last longer. That’s the issue at stake here.

Or have you ever been at the mercy of a non-active / committed library author? :)

Krzysztof Zabłocki:

A lot of cool things I built for the community or in my own apps would be impossible to do if this happens.

Adam Kaump:

+ Encourages more thoughtful API design

+ Encourages community-driven changes to frameworks that can benefit everyone

Sean Reilly:

:( re Apple’s stance on the Swift final-by-default proposal.

Mike Ash:

“I think so, Brain, but don’t people need to ship software without waiting years for us to fix the bugs they encounter?”

Steven L:

for final in swift, is there no backdoor option? ’cuz if it’s truly enforced, that’s super painful and short-sighted


I think this is one of the worst decisions in the history of language design.

Scott James Remnant:

I disagree that an open method overridden from a superclass is implicitly open.

As the rationale for the proposal states, overridability is hard to get right, and there is no guarantee that the consumer of an API is going to think about it. The default for override methods should not be open or final, it should be the internal equivalent.

John McCall:

I don’t think that defaulting to non-open would be a good idea. Like I covered in the proposal, inherited open methods remain open; letting an override implicitly close off an open method would create a pretty unfortunate error-of-omission situation.

We could remove the default here and require the method to be explicitly open/nonopen/final, but then we really do pile up the modifiers[…]

L. Mihalkovic:

So simply as a matter of following the logic of the proposal, every subsequent extension of something explicitely marked as open for subclassing should defacto be placed back into the default non-subclassable state, forcing the author to have to explicitely validate the new set of relationships existing between the methods in the subclass. Otherwise the logic does not hold, and the original proposal does not hold water[…]

Károly Lőrentey:

I’m enthusiastic about sealed-by-default classes, but to be honest, I personally have no idea what default (if any) would be best for class members. Ask me again after I’ve worked with the new classes for a couple of months.

L. Mihalkovic:

This is what i find so unique about this situation: there is not 2 months to decide, there is not a couple of implementations to compare.. there is here and now to decide for the next 20 years, with zero experience with the api and very little external references (which nobody seems to [have] any practical experience with) ... nonetheless it must all be fleshed out before the 28th. I [sincerely] hope the core team is more prepared than they currently let out.

John Randolph:

- infinity for this proposal.

If adopted, it will mean that whenever I want to extend a class implemented by a short-sighted developer, I’ll have to resort to wrappers and classes that own instances of the lobotomized class, etc, etc.

Joe Groff:

Users who are third-party framework devs I expect will benefit the most.

They don’t have Apple’s resources to deal with all the backward-compat testing to cope with unconstrained subclassing and patching.

Jonathan Hull:

I would really like to see the methods match the open-ness or final-ity of their enclosing scope by default. Note: I also feel this way about access levels, which work this way except for public… I believe they should work the same way for public too (while keeping internal as the default level for top-level structures). If a class is public open, I am typically going to want 85%+ of the methods to also be public open. Having to annotate each of them adds a lot of visual noise, and is fairly easy to forget to add when refactoring amidst that visual noise. I already have this problem with public, so I expect it will only be worse for public open.

Update (2016-07-19): @Gary_BBGames:

I like Swift, but this will be an absolute shit sandwich

Gwynne Raskind:

The reasoning from the core team that working with vendors replaces the ability to subclass to work around problems simply doesn’t hold water for me. This is a very real and very common issue, and there are endless cases where vendors won’t or even can’t solve the problem, especially in closed-source code.


In short, saying “filing a bug will work” isn’t good enough to justify locking out the ability of developers to deal with problems in vendor code. It’s simply not true - it’s certainly almost never been true of Apple frameworks, and even when it has, that doesn’t help anyone “now”. (To be clear, I’m not suggesting Apple is unresponsive to Radars. However, I am saying that there’s no transparency, no confidence in getting fixes, and no hope of any kind of reasonable (from a local perspective) timeline for deployment of fixes.)


If not for the practical considerations, I’d love it! Conceptually speaking, I find it elegant, even harmonious. But again, in practice, the result isn’t so pretty.

Juan Laube:

-1, I really think this a step in the wrong direction.

I recognise the problem around this, and why something is needed. However, I don’t like the idea of restricting things by default. In the attempt to solve a problem, we will create more problems by introducing more workarounds to replace what now is being overridden.

Brent Royal-Gordon:

You know, one thing I haven’t seen mentioned is that, just as sealed-by-default preserves the options of library programmers, it also preserves the options of the language itself.

Suppose the people who think this is a huge mistake are correct, and we ultimately conclude that sealed-by-default is a disaster. What can we do about it? Well, we change Swift 3+n to open all classes by default. This would be source- and binary-compatible with all existing code; the only wrinkle is that classes compiled with a sealed-by-default compiler would still be sealed. (And that’s not even a problem yet, since stable ABIs are not a thing yet.)

The reverse, however, is not true. Going from open-by-default to sealed-by-default is source- and binary-incompatible. If we don’t do it now, we may never be able to do it.

Tino Heth:

In reality, nobody will measure the effect of this change, and those in favor of the proposal might even call the absent of a horrible catastrophe a “proof” for their opinion.

David Owens II:

As an API author, if I need to have a value type, I cannot use inheritance. And more important, especially with regards to many of the arguments against this proposal, you cannot fix any issues with these struct types with inheritance either. It’s at this point that I find your arguments extremely weak: if it is so crucial that inheritance be enabled by default for class types, why are you not more concerned about Swift’s focus on value types and using value semantics for APIs? After all, there are pretty much no APIs in Swift’s libraries that you will be able to patch this way.

I am, and I’ve been writing about that concern from the beginning. As I said at the top, this proposal is just one piece of the larger trend.

There’s absolutely nothing in this proposal that prevents Swift from providing tools to get you access to what you need. For example, imagine a world where you could download a developer Swift module that contains all of the unoptimized code.

This is the hypothetical “escape hatch” that people have been talking about. In theory it could work, if implemented, and if you were able to get all the unoptimized code. That seems rather unlikely to me, and it would be a lot of trouble to go through for the (I think) small benefits the proposal would provide. There are so many more pressing issues I would like the Swift team to work on than building restrictions and then an escape hatch for them.

Update (2016-07-21): Benjamin Mayo:

In general, language design should not be decided by the possible existence of buggy code. However much we strive to make perfect code, there will always be bugs.

Eli Perkins (via Natasha Murashev):

I would have loved to seen this implemented as an opt-in compiler-time flag, rather than a syntactical level keyword.

Peter Livesey:

It seems like for 3rd party libraries only, we are afraid that people will subclass things when they shouldn’t? Is this a real problem? I’ve honestly never seen anyone incorrectly subclass something in a 3rd party library when there was a good alternative. Maybe I have and I forget, but it happens so rarely that I don’t think it’s a real problem to fix.

Peter Livesey:

With this proposal: consumers are screwed until the library is fixed, but the change is backwards compatible.

Without this proposal: consumers are fine to choose what to do (and maybe make mistakes), but the change could be backwards incompatible.

To me, consumers not being screwed > changes being backwards compatible.

Tal Atlas:

I have no concerns intrinsically to the behavior discussed but rather to the complexity that this brings. Creating another type of access control and possibly keyword just adds complexity to an already fairly complex language.

Nevin Brackett-Rozinsky:

I am reminded of the atomic / nonatomic situation in Objective-C. Yes, atomic is generally safer, but almost every property in almost every class is declared nonatomic. It was a mistake to set atomic as the default, which caused a profusion of noise in property declarations.

Chris Lattner:

The proposal has been returned for revision, again. :-)

As with the first round of discussion, the community generated a large number of emails, exploring the various aspects of the proposal. While many community members agree with the thrust of the proposal, a number of people are concerned with the boilerplate being introduced by the proposal, among other issues. The core team spent over two and a half hours discussing this topic from first principles, and has come up with a similar-but-different approach that should reduce the boilerplate, while still accomplishing the primary aims of the proposal. John McCall will be revising the proposal today and we’ll restart a short discussion period about it tomorrow.


Conflation of public and open; it feels like open is a new higher access level, like getting married, or going sudo or something. If this proposal was accepted, open should substitute public, and never be alongside it (the same way public private class makes no sense)

Garth Snyder:

In the original proposal (and the ensuing discussion), there was tacit agreement that subclassability/overridability and access levels should be orthogonal. However, given the direction that the design has taken since then, I think we should revisit that decision.


Third, developers already understand access levels and how they interact. If open is just an access level, all of this proposal’s changes can be fully and naturally described in one line: “public no longer includes the right to subclass or override. To get the behavior formerly known as public, use open instead.” Clear, concise, and not very controversial.

Update (2016-07-23): See also: Core Intuition.

Update (2016-07-25): Scott James Remnant:

I have no objection to the fundamental concept of the proposal, but this should not come at the cost of the language, just on the grounds of the timeline. I think it would be better for this to be re-proposed without the rush for future versions of Swift - especially since it’s “additive.”

Dmitriy Y. Volkov:

I’ve been reading mailing list for the last two reviews of this proposal and discussion turned from “We shouldn’t add this at all” - which was kind of justified to “How actually will it interops with other swift features” - which still has a lot of questions of how exactly this modifiers will play out with other accessibility modifiers, and it seems like there is still no single answer. As Scott said, I have the same feeling that this proposal is being rushed to be accepted before the changed to Swift 3 are locked, and this won’t do any good to its semantics.

Gwynne Raskind:

This is one of the most contentious topics I’ve seen come across Swift so far. While there may be no stopping the avalanche now as far as implementing the proposal in general (which I remain against, though many of the arguments I’ve heard in favor of it are starting to shake my conviction!), it definitely begs more consideration than running up hard against a release deadline. Goes double considering the disagreement even among the proposal’s supporters about the best semantics.

Update (2016-07-29): Chris Lattner:

The third review of “SE-0177: Allow distinguishing between public access and public overridability” ran from Active review July 21...25. The proposal has been accepted with revisions.

This proposal was far better received by the community than previous versions of the proposal, and the “first design” was the favored path within it. However, there were some concerns raised about the complexity of the model, stemming from non-obvious combinations like “open private”. As such, the core team has requested that the proposal be revised to make “open” function as another access control specifier. “open” is now simply “more public than public”, providing a very simple and clean model.

John has already revised the proposal to the new model, I encourage you to read it if you haven’t already.

Here’s the final version of the proposal.

Friday, July 15, 2016

Lepton Image Compression

Dropbox (Slashdot):

We are pleased to announce the open source release of Lepton, our new streaming image compression format, under the Apache license.

Lepton achieves a 22% savings reduction for existing JPEG images, by predicting coefficients in JPEG blocks and feeding those predictions as context into an arithmetic coder. Lepton preserves the original file bit-for-bit perfectly. It compresses JPEG files at a rate of 5 megabytes per second and decodes them back to the original bits at 15 megabytes per second, securely, deterministically, and in under 24 megabytes of memory.

We have used Lepton to encode 16 billion images saved to Dropbox, and are rapidly recoding our older images. Lepton has already saved Dropbox multiple petabytes of space.

See also: StuffIt X.

Improving Color on the Web

Dean Jackson:

The past few years have seen a dramatic improvement in display technology. First it was the upgrade to higher-resolution screens, starting with mobile devices and then desktops and laptops. Web developers had to understand high-DPI and know how to implement page designs that used this extra resolution. The next revolutionary improvement in displays is happening now: better color reproduction. Here I’ll explain what that means, and how you, the Web developer, can detect such displays and provide a better experience for your users.


The Web has often struggled to handle colors correctly. I’m sure there are some readers out there who painfully remember Web-safe colors! While we’ve moved on from that, we still have limitations, such as HTML and CSS having been defined to work only in the sRGB color space. Just like the example of hober’s shoes above, this means there are many colors that your CSS, images, and canvas are unable to represent.


WebKit now supports the (new to CSS Color Level 4) color-gamut media query.


This is what members of the WebKit project have proposed for CSS. The current idea is to add a new function called color() that can take a color profile as well as the parameters defining the color.

Gus Mueller:

This writeup is an incredibly great explanation of wide gamut issues with a touch on deep color as well. If you are a developer who uses color in any way, you’re going to want to read this. Wide gamut displays are already here.

Previously: iPad Pro, True Tone, and Color Gamut.

Running System 1 on El Capitan

Rich Trouton:

As part of some research which I’m doing on Mac filesystems, I wanted to see if it was possible to get Apple’s System 1 running on OS X El Capitan. The reason that I am specifically interested in System 1 is that this OS used Apple’s shortest-lived filesystem: Macintosh File System.

After discussing the issue, my colleague @mikeymikey pointed me in the direction of using the Mini vMac emulator application to accomplish this. He also pointed me towards the correct places where I could download a System 1 disk image and the needed Mac ROM file.

The Windows Shutdown Crapfest

Moishe Lettvin (2006, via @SwiftOnSecurity):

The most frustrating year of those seven was the year I spent working on Windows Vista, which was called Longhorn at the time. I spent a full year working on a feature which should’ve been designed, implemented and tested in a week. To my happy surprise (where “happy” is the freude in schadenfreude), Joel Spolsky wrote an article about my feature.

I would like to try to explain how this happened.


So that nets us an estimate -- to pull a number out of the air -- of 24 people involved in this feature. Also each team was separated by 6 layers of management from the leads, so let’s add them in too, giving us 24 + (6 * 3) + 1 (the shared manager) 43 total people with a voice in this feature. Twenty-four of them were connected sorta closely to the code, and of those twenty four there were exactly zero with final say in how the feature worked. Somewhere in those other 19 was somebody who did have final say but who that was I have no idea since when I left the team -- after a year -- there was still no decision about exactly how this feature would work.

Thursday, July 14, 2016

Facebook’s Mobile Device Testing Lab

Frederic Lardinois (via Peter Steinberger):

The mobile device lab currently occupies 60 racks in the data center. Each rack holds 32 phones, for a total of almost 2,000 devices, but, as Facebook’s Antoine Reversat told us, the plan is to bring this number to 64 devices per rack.

Each rack features its own Wi-Fi signal and is also an EMI enclosure to make sure that neighboring racks can’t pick up the Wi-Fi signals from its neighbors.

Every time an engineer makes changes to one of Facebook’s main mobile apps, that new version of the app is automatically tested on these devices to ensure that there are no crashes or performance issues.


All of the phones also need to be connected to a PC or Mac in order to receive the latest code. Because of this, there are PCs and Mac Mini’s under every rack. Facebook uses eight Mac Minis per rack for iOS testing (because each one can only talk to four iPhones) or four OCP Leopard servers for testing Android devices.

Legal Decisions

Orin Kerr:

The U.S. Court of Appeals for the 9th Circuit has handed down a very important decision on the Computer Fraud and Abuse Act, Facebook v. Vachani, which I flagged just last week. For those of us worried about broad readings of the Computer Fraud and Abuse Act, the decision is quite troubling. Its reasoning appears to be very broad. If I’m reading it correctly, it says that if you tell people not to visit your website, and they do it anyway knowing you disapprove, they’re committing a federal crime of accessing your computer without authorization.


At this point you may be thinking: Hey, wait, didn’t the en banc 9th Circuit rule in Nosal I that using a computer in violation of its terms of use is not a CFAA violation? If intentionally using a computer in violation of the terms of use is legal authorized access, as the en banc 9th Circuit held in Nosal I, why is intentionally using a computer after receiving a cease-and-desist letter criminal access “without authorization”? In one case, the user goes to the website and sees the terms; in the other, the website owner contacts the user and shows the terms to them. But it’s the same thing, right?

Bruce Schneier:

In a truly terrible ruling, the US 9th Circuit Court ruled that using someone else’s password with their permission but without the permission of the site owner is a federal crime.

Mark Rumold:

In a dangerously flawed decision unsealed today, a federal district court in Virginia ruled that a criminal defendant has no “reasonable expectation of privacy” in his personal computer, located inside his home. According to the court, the federal government does not need a warrant to hack into an individual’s computer.

MathML Improvements in WebKit

Frédéric Wang:

This new feature is obvious: You can now create a hyperlink for any part of a mathematical formula!


In the following screenshot, you can see that the letters f, x and y are now drawn with this special mathematical italic glyphs and that WebKit uses the conventional fraktur style for the Lie algebra g. Note that the prime is still too small because WebKit does not make use of the ssty feature yet.


As said in my previous blog post, the rendering of large and stretchy operators have been rewritten a lot and as a consequence the rendering has improved. Also, I mentioned that the width of operators may depend on their height. This may cause accumulated approximations during the computation of preferred widths. The old flexbox-based implementation incorrectly forced layout during preferred computation to avoid that but a quick workaround for that security concern caused the approximate preferred widths to be used for the logical widths. With our new implementation, the logical width is now correctly calculated. Finally, we added partial support for the mpadded element which is often used to tweak spacing in mathematical formulas.

Update (2016-07-24): See also:

Pinboard Turns Seven

Maciej Cegłowski (via Manton Reece):

I’ve added revenue this year because I’m no longer afraid of competitors, and I’d like to encourage people who are considering doing their own one- or zero-person business. The site costs something like $17K/year to run, so you can make a good living at this artisanal SaaS stuff.

As you can see, most everything has been steady year-to-year for a while now. Revenue dropped a bit in 2015 (as I moved to an annual subscription system), then picked up substantially this year as the first wave of subscription renewals came due, and people had the option of renewing for multiple years.

Wednesday, July 13, 2016

Mailbox Names via IMAP

Bron Gondwana:

So IMAP supports arbitrary hierarchy separators. In practice only / and . appear to be common, though I’m sure there are still systems out there using \ which brings its own special hell in C-like programming languages which use \ as an escape character (as anybody who’s ever programmed for Windows will know).

Some operating systems use case significant file systems, others don’t. So on some systems, README.TXT and Readme.txt are the same file, but on others they aren’t. Of course IMAP chose to leave case sensitivity up to the implementation... sort of.


I spent a lot of time workshopping our options, and in the end the RFC requirement that ALLCAPS INBOX had to match LIST and the fact that every other server always returns it as INBOX in that case convinced me that the folders really had to be subfolders of ALLCAPS INBOX, rather than using Inbox as the name. Also, any other spelling of INBOX needed to not be allowed at the top level, because of the case insensivity rule.

Apple Faces Patent Lawsuit Over iPhone’s Battery Technologies

Joe Rossignol:

Somaltus, LLC has filed a complaint against Apple today in an Eastern Texas district court, accusing the iPhone maker of infringing upon its 2010 patent related to complex battery technologies. The small Frisco, Texas-based firm also filed lawsuits against Asus, Lenovo, Samsung, Sony, and Toshiba over the same patent.


Specifically, it appears that the infringement claim at least partially relates to the iPhone’s process of charging in fast-charge mode until the battery reaches 80% capacity, and then adjusting to trickle-charge mode above 80% capacity.

Wasn’t Apple doing that with PowerBooks in the 90s?

Swift Playgrounds Aren’t HyperCard

Adam Banks:

HyperCard, “like a software erector set,” would crystallise computing into building blocks that any user could snap together to implement the functionality and user interface they had in mind. That’s not what Swift Playgrounds does today, however. Apple’s newest attempt to democratize coding presents a very inviting experience to the budding developer, but it insists that you code. And even after having done so, you still don’t get a deliverable app—only a work in progress.


Keith Martin, a senior lecturer at the University of the Arts London (UAL) with a long-term interest in interactive development, also wanted Apple to think beyond code. “Swift Playgrounds seems to be a tacit admission that Swift is not easy. The trouble is, it doesn’t actually address the fundamental issues, it just tries to show stuff in a cutesy form.” The difficult concepts that made learning Objective-C (Apple’s favoured language before Swift) feel like “banging my head against a wall” are still unavoidable.


Yet while hypertext went everywhere, programming without coding went nowhere.


As [Douglas] Adams recognised, HyperCard didn’t dumb down; it simplified up. The way it shoehorned objects and classes into a mundane Rolodex metaphor made advanced principles concrete, so users could bash them around and see for themselves what they achieved instead of having to learn dry theory before they could even start on syntax.

Tuesday, July 12, 2016

Amazon’s Chinese Counterfeit Problem Is Getting Worse

Ari Levy (via John Gordon and John Gruber):

In May, reported on a Facebook group, now consisting of over 600 people, whose members have seen their designs for t-shirts, coffee mugs and iPhone cases show up on Amazon at a fraction of the price of the originals. The designers described it as a game of whack-a-mole, where fakes pop up more quickly than they’re taken down.


To unsuspecting consumers, fake products can appear legitimate because of the Fulfillment by Amazon program, which lets manufacturers send their goods to Amazon’s fulfillment centers and hand over a bigger commission, gaining the stamp of approval that comes with an FBA tag.


Making matters worse, when buyers unhappy with the cheaper alternatives leave a bad review, it drags down Bergman’s standing because the reviews are all thrown together.


As a marketplace, Amazon isn’t legally responsible for keeping counterfeit material off the site as long as it responds to complaints and takes action when it’s brought to the company’s attention.

Amazon faces a sort of strategy tax.

Update (2016-07-21): Ari Levy (via John Gruber):

Birkenstock is walking away from

Plagued by counterfeits and unauthorized selling on the online shopping site, the sandals company will no longer supply products to Amazon in the U.S. starting Jan. 1. Additionally, Birkenstock won’t authorize third-party merchants to sell on the site, according to a letter the company sent to several thousand retail partners on July 5.


The only way to get Amazon’s support in creating a clean environment, according to Kahan, is by selling the entire catalog to Amazon. It’s part of the online retailer’s effort to be the one-stop shop for anything and everything.


Birkenstock will be telling consumers to purchase only from authorized retailers, and that any products listed on Amazon can’t be trusted. “So, buyer beware,” he wrote.

Update (2016-08-19): Nick Heer:

It’s not just counterfeit goods on Amazon that are a problem for legitimate businesses. Jason Feifer writes for Entrepreneur magazine on a new scheme that marries Amazon’s generally low prices and dubious eBay sellers[…]

Monday, July 11, 2016

The Pokémon Go Phenomenon

Serenity Caldwell et al.:

Most people have at least heard of Pokémon — Nintendo’s ever-popular title — which asks players to travel a fictional world in order to collect every creature out there. But today’s world is not the world of the 1990s: Nintendo and Niantic Labs have teamed up to let players catch Pokémon in the very world we live in, thanks to a combination of GPS, augmented reality, and dorky-cute graphics.

A good introduction for people like me who have basically only heard about Pokémon—until this weekend—through Swift example code.

James Vincent (Hacker News):

Nintendo’s stock continues to skyrocket following the release of Pokémon Go. After an increase of 9.3 percent with the game’s launch last week, the company’s share price rose 24.52 percent on Monday to ¥20,260 ($193) — its highest one-day surge since 1983, adding $7.5 billion to the firm’s market value. The game has topped app download charts in the US, Australia, and New Zealand, and according to some market researchers, has already been installed on 5 percent of all Android smartphones in America.

Mitchel Broussard:

Because it requires players to travel to real-world destinations in order to stock up on Poké Balls, eggs, potions, and compete at gyms, some individuals have been capitalizing on the game’s mechanics to trap and rob its players. According to a Facebook post from the O’Fallon Police Department in Missouri, four people were arrested over the weekend after using a Lure Module at a PokéStop to draw in unsuspecting players and rob them at gunpoint.

Aaron Levie:

Nintendo’s mobile strategy:
2008: what’s an iPhone
2010: what’s an Android
2012: nope
2014: still no
2016: change how society functions

Although it sounds like Nintendo’s contribution was mostly the IP. The game was developed by Niantic, of Ingress fame.

Previously: Nintendo.

Update (2016-07-11): Adam Reeve (via David Chartier):

To play the game you need an account. Weirdly, Niantic won’t let you just create one - you need to sign in with an existing account from one of two services - the website or Google. Now the Pokemon site is for some reason not accepting new signups right now so if you’re not already registered there you’ll need to use a Google account - and that’s where the fun begins.


Let me be clear - Pokemon Go and Niantic can now:

  • Read all your email
  • Send email as you
  • Access all your Google drive documents (including deleting them)
  • Look at your search history and your Maps navigation history
  • Access any private photos you may store in Google Photos
  • And a whole lot more

What’s more, given the use of email as an authentication mechanism (think “Forgot password” links) they now have a pretty good chance of gaining access to your accounts on other sites too.

Update (2016-07-12): @SwiftOnSecurity:

When you give a company keys to your account, you’re giving your keys to their employees and anyone who hacks them. Not a question of trust.

Yan Zhu:

weird thing abt PokemonGo getting full goog account access is that iOS showed no permission dialog. goog should just revoke their app tokens

Adam Reeve has posted a FAQ.

Rosyna Keller:

I don’t understand… @NianticLabs fixed the Google scope bug in Ingress on April 19th but left it in Pokémon GO?!

William Turton:

But in a call with Gizmodo, Reeve backtracked his claims, saying he wasn’t “100 percent sure” his blog post was true. On the call, Reeve also admitted that he had never built an application that uses Google account permissions, and had never tested the claims he makes in the post.

Cybersecurity expert and CEO of Trail of Bits Dan Guido has also cast serious doubt on Reeve’s claim, saying Google tech support told him “full account access” does not mean a third party can read or send or send email, access your files or anything else Reeve claimed. It means Niantic can only read biographical information like email address and phone number.


A product security developer at Slack tested the token provided by Pokémon Go and found that it was never able to get data from services like Gmail or Calendar.

Nicole Lee:

Niantic Labs and The Pokémon Company issued a response to Engadget, confirming that it’s not actually reading your emails. Still, it has far more access than is necessary for the game and the company says that while it’s working on a fix for the client to only request the correct permission, Google will reduce Pokémon Go’s access on its end ‘soon.’

Update (2016-07-13): Lukas Mathis:

Much of the argument for Nintendo to create mobile games was based on the idea that Nintendo could not survive just selling its own consoles. I think it’s fair to say that the last few years have shown this to be false. Even with the Wii U being an abysmal failure, Nintendo is consistently profitable.


For Pokémon GO to achieve the same kinds of numbers, it will have to continue doing this well for quite a while.

That’s not to say that Pokémon GO can’t achieve those numbers - but how many of Nintendo’s other games will be lottery wins the size of Pokémon GO?


One final point: it feels a little bit disheartening to see the adulation Nintendo is currently getting for moving towards the same kind of manipulative, free-to-play, IAP-monetized games we get from tons of other rather questionable mobile gaming companies. It really took a very short amount of time for us to accept these games as the new normal.

Mitchel Broussard:

One week after it launched in the United States, Pokémon Go is now the biggest mobile game in the country’s history, in statistics focusing on the amount of daily active users (DAU) the app draws.

Update (2016-07-15): See also: Aeriform.

Update (2016-07-25): See also: Free-to-Play Dogs (via Accidental Tech Podcast).

Ben Lovejoy:

After Nintendo’s market-cap more than doubled following the launch of the game, investors seem to have belatedly realized that the company sees only indirect benefits from Go. Bloomberg reports that shares plunged 18% – the maximum allowed before trading is automatically halted by the Tokyo stock exchange – with a further drop looking likely when trading resumes.

Update (2016-07-28): Pokémon Free-to-Play Dogs (via Accidental Tech Podcast).

Apple and the Blind

Katie Dupree (via John Gruber):

The company, for example, made the first touchscreen device accessible to the blind via VoiceOver. Recent announcements of Siri coming to Mac this fall, and of newer innovations, like a magnifying glass feature for low-vision users, have continued the promise of improving the Apple experience for those who are blind and low vision.


The most recent example of community-driven innovation can be found on the Apple Watch. During a meeting, Herrlinger explains, a person who sees could easily peer down at their watch to keep an eye on the clock. A person who is blind, however, hasn’t had a way to tell time without VoiceOver.

After confronting the conundrum, Apple solved the issue by making a feature that tells time through vibrations. The addition, Herrlinger says, is coming to watchOS 3 this fall.


For Castor, Braille is crucial to her innovative work at Apple — and she insists tech is complementary to Braille, not a replacement.

“I use a Braille display every time I write a piece of code,” she says. “Braille allows me to know what the code feels like.”

In coding, she uses a combination of Nemeth Braille — or “math Braille” — and Alphabetic Braille. Castor even says that with the heavy presence of tech in her life, she still prefers to read meeting agendas in Braille.

The History of “This Web Site Is Well-crafted” Hints

Adrian Holovaty:

Over time, technology stabilizes and the techniques become expected. 10+ years ago, in the era of .cgi and .asp, I remember geeking out with Simon Willison about beautiful URL structures we’d seen. No file extensions! Readable! Hackable!

To us, they were signals that a web development team sweated the small stuff. It’s like the famous Steve Jobs story about making the inside of the hardware look just as nice as the outside, even if nobody ever sees it, because you have pride in your work.

With this in mind, I put together a list of these hints, as I remember them. Perhaps it’s of some historical interest, or maybe it’s just fun nostalgia. Almost all of these have become mainstream by now.

Praise for the Fujitsu ScanSnap

Kirk McElhearn:

If I had known, I would have opted for the more expensive ScanSnap iX500 (, Amazon UK), which has a larger document feeder, and wifi, and is faster. And I’ve thought from time to time that, if mine dies, I’ll replace it with the better model, but it seems to be a workhorse. It’s rare that I have a lot of documents to scan, and it’s no hassle to have the device connected to my Mac.

But, for now, this is one of the best pieces of hardware I’ve ever bought. It’s an essential tool for a modern home or office, where we are inundated with paper.

I tried many document scanners over the years, but none worked reliably. I wanted to go paperless, but not if the hardware was always getting in the way. About nine years ago, I got a ScanSnap S500m. It seemed expensive, but it ended up being one of my best purchases ever. It’s so easy to use, almost always does the right thing, and mine is still going strong today—although the plastic has yellowed like a Mac 128K.

Friday, July 8, 2016

Continuous: C# and F# IDE for iPad

Frank A. Krueger (via Federico Viticci):

Continuous gives you the power of a traditional desktop .NET IDE - full C# 6 and F# 4 language support with semantic highlighting and code completion - while also featuring live code execution so you don’t have to wait around for code to compile and run. Continuous works completely offline so you get super fast compiles and your code is secure.

Continuous gives you access to all of .NET’s standard library, F#’s core library, all of Xamarin’s iOS binding, and Xamarin.Forms. Access to all of these libraries means you won’t be constrained by Continuous - you can write code exactly as you’re used to.

Switching to Apple’s Two-Factor Authentication

Dan Moren:

Apple has, for a while now, offered two separate additional security measures to protect your Macs, iOS devices, and iCloud account, but thanks to some inexpert nomenclature, it can be a little difficult to tell them apart.

The first, two-step verification, has been offered for several years. It prompts you to enter a four-digit code when you sign into your iCloud account, purchase something from one of Apple’s stores on a new device, or make changes to your Apple ID. Those codes were delivered by push notification to an authenticated device of your choosing, or via SMS text message.


The newer two-factor authentication is an improvement upon that process, which Apple started rolling out last year. While the principle is similar, the execution is refined. The verification code is now six digits and is automatically sent to all of your authorized devices.


Though Apple didn’t provide an obvious way to make that jump, the key is simply to deactivate your existing two-step authentication[…]

Dan Moren:

If you want to see a list of trusted devices and those that can receive two-factor codes (which largely but not entirely overlap), go to the iCloud preference pane on your Mac, or the iCloud section of Settings on your iOS device, and look at your account, then select Devices. That’ll provide a list of every device logged into your iCloud account; selecting each will tell you if they’re trusted and can receive two-factor codes.


As mentioned in a subsequent update, two-factor authentication does not remove the need for app-specific passwords, but it does seem that you no longer need them for any Apple services.

Mozilla-Yahoo Contract Clause

Kara Swisher (via Hacker News):

Under terms of a contract that has been seen by Recode, whoever acquires Yahoo might have to pay Mozilla annual payments of $375 million through 2019 if it does not think the buyer is one it wants to work with and walks away.

That’s according to a clause in the Silicon Valley giant’s official agreement with the browser maker that CEO Marissa Mayer struck in late 2014 to become the default search engine on the well-known Firefox browser in the U.S.

Mozilla switched to Yahoo from Google after Mayer offered a much more lucrative deal that included what potential buyers of Yahoo say is an unprecedented term to protect Mozilla in a change-of-control scenario.

Slower by Design

Mark Wilson:

The short answer is no. Facebook actually slows down its interface to make users feel safe, a Facebook spokesperson confirmed in an email. “While our systems perform these checks at a much faster speed than people can actually see, it’s important that they understand what we do behind the scenes to protect their Facebook account,” the spokesperson wrote. “UX can be a powerful education tool and walking people through this process at a slower speed allows us to provide a better explanation and an opportunity for people to review and understand each step along the way.”

If half of Facebook’s billion users spend 5 seconds waiting on this check, that’s 694,444 hours, or 28,935 days of collective time lost. But Facebook isn’t alone. Websites and apps now operate on the magnitude of milliseconds. But such speed can make users skeptical or even confused, so companies are responding by building slower, more deliberate interfaces. Wells Fargo admitted to slowing down its app’s retinal scanners, because customers didn’t realize they worked otherwise, while various services on the web including travel sites, mortgage engines, and security checks are all making a conscious effort to slow down their omnipotent minds because our puny human brains expect things to take longer.

Why is it that Kayak has to slow down its search so people believe it’s working hard to find them the best deals, but no one doubts Google’s quick search results?

Thursday, July 7, 2016

Protecting Your Network From Photos Uploads

Adam C. Engst (May 2015):

My Internet connection runs at 30 Mbps down and 5 Mbps up, and when I turned on iCloud Photo Library for the first time, Photos completely destroyed Internet performance for every device in the house: Web pages loaded slowly, Google Hangouts struggled, Netflix buffered repeatedly, Rdio stuttered, and even Dark Sky on the iPhone timed out getting climate data. I had to promise to pause syncing whenever Tristan needed to do homework (which is whenever he’s home, it seems), and it was clearly something that couldn’t run during our work days.

I recently enabled iCloud Photo Library for a small library in our household, and it did more than reduce the Internet performance. It made all our Macs and devices stop working. Web pages immediately showed errors. Mail and Dropbox wouldn’t sync at all. Apple TV couldn’t play anything. Wi-Fi calling didn’t work. This is with a pretty fast cable connection. Oh, and quitting Photos doesn’t provide relief because uploads continue via a helper process.

Every other network-intensive app that I use—Arq, CrashPlan, Dropbox, etc.—includes a bandwidth limiter. (Apps that upload seem to cause more problems than ones that download.) Photos has no throttle, just an off switch, which doesn’t really help because neither of its settings does what I want. Let it run, and no other app on any device can realistically access the Internet. Turn it off, and the photos never upload. It boggles the mind that the app was released this way and not fixed in the 15 months since (or, seemingly, in Sierra).

The standard advice is to install the Network Link Conditioner and use it to limit the bandwidth available to your Mac. To do this, you need to be a registered developer. Then you can download the “Hardware IO Tools for Xcode 7.3” and double-click the Network Link Conditioner.prefPane.

Note that if your Mac’s login account is a Standard rather than Admin user, you will not be able to click the button to enable Network Link Conditioner after installing it. System Preferences will beachball forever. However, because it affects the entire Mac, you can enable it from an Admin account and it will still limit the bandwidth of Photos running in another account. Even once it’s installed and enabled, Network Link Conditioner won’t work if you boot the Mac and log into a Standard account. You have to first log into an Admin account in order for it to load properly.

The problem with using Network Link Conditioner is that the network will also be slowed down for every other process on the Mac. Also, Photos may still monopolize the throttled connection and prevent other apps from doing anything. To fix this you would need to use a more involved method to limit the bandwidth for a particular process (via Rosyna Keller). I have not tried this, but it looks like that process would be cloudd and that it handles more than just Photos syncing, so this would still be an imperfect solution.

Previously: iCloud Photo Library: the Missing FAQ, More Problems With iCloud Photo Library Uploads.

And, speaking of Photos, Glenn Fleishman:

Yup, that’s right: hidden photos appear in plain sight in albums and the main view in Photos. Media marked hidden disappears only from Moments, Collections, and Years. If you use iCloud Photo Library, the same is true, because the photos or videos you’ve hidden need to sync among your devices so that they are in the same state on all of them.

Update (2016-07-08): Garrett Murray:

Had the same problem when Photos first came out. Took 3 weeks of night-only NLC uploading on my Mac to finish initial sync.

When “No Longer Available” Strikes in iCloud Music Library

Kirk McElhearn:

But things get complicated when music that you have added to your iCloud Music Library from Apple Music is pulled. Labels can withdraw the right to stream certain songs and albums at any time, but you won’t be notified. You may see albums and songs in your library, but their titles are a slightly lighter color (depending on the view), and their iCloud status is No Longer Available.


In most cases, this is little more than an annoyance. But there are certain situations where this availability is an iCloud Music Library problem. Look at the screenshot below: it’s my iTunes library in Albums view. You’ll see that there are two instances of Chicago’s At Carnegie Hall. This is because the first one I added to my library suddenly became No Longer Available. When I searched for the album, I found it and added it to my library, and the only difference is the addition of (Live) after its title. Somehow, the record label tweaked the name of the album, and it was considered to be a totally different album.

He also has a bunch of “No Longer Available” tracks with missing metadata.

Update (2016-07-23): Christopher Thielen (via e-mail):

With macOS Sierra’s “Optimize Storage” removing previously watched iTunes purchases, what do you think would happen if the movie is no longer available for download?

The iTunes Terms and Conditions recommends users create a backup but even a Time Machine backup would lose the file should it be off the main computer (“optimized”) long enough.

Android Flash Keyboard Hijacks Lock Screen, Violates Privacy

SecurityWeek News (via @SwiftOnSecurity):

A third-party keyboard application for Android that had over 50 million installs was found to collect user data and send it to a remote server, Pentest Limited researchers reveal.


Right from the start, however, Flash Keyboard raises a red flag, given that it asks for a great deal of permissions that it isn’t supposed to have. It can run at startup, can read and write home settings and shortcuts, can use network and Bluetooth as it likes, can modify system settings, disable the lock screen, force-stop other applications, and read the status of phone, user ID, and more. […] Moreover, Flash Keyboard uses device admin APIs that allow it to replace the standard Android lock screen with its own custom lock screen, which is monetized by displaying custom ads.


The researchers discovered that the application was communicating with servers in several countries, including the United States, the Netherlands, and China, and that it sent the following information to them: device manufacturer and model number, IMEI, Android version, user email address, Wi-Fi SSID, Wi-Fi MAC, mobile network, GPS co-ordinates, information about nearby Bluetooth devices, and details of any proxies used by the device.

Concurrent Core Data, Now Easier Than Ever

Arkadiusz Holko:

Our main thread-bounded operations use viewContext. Background work can be performed on a context returned by the factory method newBackgroundContext(). However, usage of performBackgroundTask(_:) is recommended because of the under-the-hood optimizations.

All contexts are independent of each other, there are no parent/child relationships here. To receive changes from other contexts we simply set automaticallyMergesChangesFromParent flag to true on a context. Despite its name (*FromParent*), it works correctly with the NSPersistentContainer setup in which contexts are siblings:

Not sure I understand that last bit.

Update (2016-07-07): James O’Leary says that, despite the name of the property, it treats the persistent store itself as an implicit parent context.

Tuesday, July 5, 2016

BBEdit 11.6

Bare Bones Software:

At the end of the 30-day evaluation period, BBEdit will remain permanently functional with a revised feature set that includes its powerful text editing capabilities but not its web authoring tools or other exclusive features. BBEdit’s exclusive features may be re-enabled at any time with a purchased license.


There is a new command-line tool: bbresults. This tool reads data from stdin which is expected to be typically formed Unix error messages. This data is then used to create a BBEdit results window which provides navigation of errors and warnings within BBEdit itself.


Added Toothpaste as a factory-supplied color scheme.


When an untitled document is active in a window, the window’s entry in the Window menu will be annotated with some text derived from the beginning of the document’s contents.

I also like the new toolbar design, which is more compact and shows the document’s full path (with tilde).

Update (2016-07-08): John Gruber:

In the old days, there was BBEdit (paid) and BBEdit Lite (free). Then BBEdit Lite went away. Then we got TextWrangler, which was free.

I like this setup better. Download BBEdit for free. Use all of its features free for 30 days. After that, you still have a terrific Mac text editor, free of charge. But as soon as you need it, you can pay to enable the full feature set.

Monday, July 4, 2016

Jefferson’s Pull Request

Thomas Jefferson (via Patrick Woods):

Lots of good discussion from everyone!

Biggest changes:

  • Weeding out some of my verboseness (which, although lovely, is a bit superfluous)
  • Remove some of the focus on current events, especially at the end, and make it feel more timeless
    • We hope to keep these states for longer than just a couple years, right?

Sunday, July 3, 2016

Building My $1,200 Hackintosh

Felix Schwarz:

Remember when building a #Hackintosh was about getting a cheap “Mac”?

Today it’s about getting #macOS-running hardware that’s not outdated.

Mike Rundle (tweet, Hacker News):

To be honest, I hadn’t thought about the Hackintosh community in years, I actually forgot it was still a thing. Ian said the community was now organized around a website called and it had hardware guides, build tutorials, forums, lots of updates, and had been extremely lively in the past 18 months or so as it’s now easier than ever to build a Hackintosh. When he told me how fast his custom Mac was (faster than any iMac and most Mac Pros), and how little it cost (around $1,200–1,300) it struck me as impossible. I know that Apple hasn’t updated their MacBook Pros or Mac Pros in a long time, and I know there’s an “Apple Tax” you pay when parts like RAM or a processor are included in an Apple-designed computer, but the more we talked about his build the more excited I became. It was as if someone told me, yeah, duh, of course there are flying cars, check out my flying car over in the parking lot. You want a flying car, too?


64GB of RAM running at DDR4 speed for $229? Are you serious? I was originally looking to get 32GB but it was only about $90 more to double the amount and go full throttle. By the way, Apple charges $1,200 to configure a Mac Pro with 64GB of RAM, and those are slower DDR3 sticks. That’s almost as much as this entire system.

Granted, the Mac Pro has ECC RAM.

Not everything is rosy in Hackintosh land though, so here’s a call-out to some issues I encountered[…]


Here’s a comparison of my Hackintosh’s Geekbench runs compared to all other Macs. The multi-core score isn’t surprising considering my CPU only has 4 cores in it, but it does almost beat a 6-core Mac Pro. And the single core test shows my system is faster in day-to-day usage than any Mac that Apple makes.

After running the Cinebench GPU benchmark test, here’s the result: dramatically faster graphics capabilities than even the most high-end 12-core Mac Pro with dual D700 cards, which is a little crazy considering that machine costs almost $7,000 more than mine.

Note, also, that this violates Mac OS X’s EULA.

Update (2016-07-07): Peter Steinberger:

Needs new mac. Looks at buyers guide. Oh. :(

Update (2016-07-13): Nick Heer:

Apple’s sales decline is an 8.3% reduction compared to the year-ago quarter. Given that the most recent Macintosh news — the discontinuation of the Thunderbolt Display notwithstanding — was a spec bump of the MacBook, this is completely unsurprising. MacRumors’ own buyers’ guide shows a “Don’t Buy” indicator below every Mac except the MacBook.

Of the current lineup, fully half of all Macs — the Mac Pro, the Retina MacBook Pro, and the MacBook Air — are the most stale that those products have ever been.

Update (2016-07-17): Sebastiaan de With:

I am currently using a Hackintosh with dual-GPUs. Would rather give Apple my money.

Chuq Von Rospach:

So if you’re Apple, you’ve likely planned your product line around a new Thunderbolt 3 display that’s fully retina and has at least a 4K screen in it — and perhaps that embedded GPU so it can be used by less powerful computers. And your Macbook pros will have 2 (or 3 or 4) ports that will take either USB-C or Thunderbolt 3 and drive these monitors.

And none of that can happen without the Intel parts.

Update (2016-08-12): See also: Hacker News.

Eye-Fi Demonstrates the Danger of Cloud-Dependent Hardware

Glenn Fleishman:

Are you concerned about the long-term viability of devices that fall under the loose notion of the Internet of Things (IoT), in which a cloud connection is required to keep devices functioning? If so, a move by Eye-Fi, which makes SD cards that could automatically transfer newly taken photos to a cloud service, should give you pause. It sure did for me.

In September 2016, the company will discontinue support for features and Web apps for every model of card it shipped from its launch in 2007 up to the X2 models introduced in 2011 and sold through retail channels until March 2015. Only its Mobi line, introduced in 2013, will retain active support. Eye-Fi said in an email to customers the impetus for this move is the obsolescence of “Internet security and authentication mechanisms that were state-of-the-art in 2007 when we built them into our products but have since proven to be vulnerable.” Because these are hardware problems, the company can’t just push firmware upgrades out.


But while Eye-Fi says it began to phase out the last products that are affected starting in 2012, it allowed them to remain in retail sales channels until March 2015.

Android’s Full-disk Encryption Just Got Much Weaker

Dan Goodin:

A blog post published Thursday revealed that in stark contrast to the iPhone’s iOS, Qualcomm-powered Android devices store the disk encryption keys in software. That leaves the keys vulnerable to a variety of attacks that can pull a key off a device. From there, the key can be loaded onto a server cluster, field-programmable gate array, or supercomputer that has been optimized for super-fast password cracking.


But researchers from two-factor authentication service Duo Security told Ars that an estimated 37 percent of all the Android phones that use the Duo app remain susceptible to the attack because they have yet to receive the patches. The lack of updates is the result of restrictions imposed by manufacturers or carriers that prevent end users from installing updates released by Google.

What’s more, Gal Beniamini, the independent Israeli researcher who authored the blog post and wrote the exploit code, said that many Android devices that were once vulnerable but later patched—including a Nexus 6 he tested—can be rolled back to their earlier, unprotected state. He suspects the reversion is possible if a device has an unlocked, or unlockable, bootloader.


Beniamini said Android phones have a similar silicon-bound key dubbed SHK that’s used for some cryptographic functions. But rather than using the SHK to directly unlock an encrypted drive, the Qualcomm TrustZone uses the SHK to create a second key that exists as a software variable. It’s this second key that can be extracted through one of the methods outlined above.

How AWS Came to Be

Ron Miller (Hacker News):

What you may not know is that the roots for the idea of AWS go back to the 2000 timeframe when Amazon was a far different company than it is today — simply an e-commerce company struggling with scale problems. Those issues forced the company to build some solid internal systems to deal with the hyper growth it was experiencing — and that laid the foundation for what would become AWS.

Speaking recently at an event in Washington, DC, AWS CEO Andy Jassy, who has been there from the beginning, explained how these core systems developed out of need over a three-year period beginning in 2000, and, before they knew it, without any real planning, they had the makings of a business that would become AWS.

See also: The Secret to Amazon’s Success: Internal APIs, Steve Yegge’s Google Platforms Rant.

Saturday, July 2, 2016

Dash 3.3

Great update to one of my favorite apps. Unlike before, it doesn’t work with Xcode’s documentation out of the box. You need to download the new Apple API Reference docket. But then you get access to Apple’s new, unified documentation, instead of having a separate set for each platform.

See also: Dash, Xcode 8 and macOS Sierra.

How Oracle’s Business As Usual Is Threatening to Kill Java

Sean Gallagher:

Stop me if you’ve heard this one before: Oracle has quietly pulled funding and development efforts away from a community-driven technology where customers and partners have invested time and code. It all seems to be happening for no reason other than the tech isn’t currently printing money.


Oracle employees that worked on Java EE have told others in the community that they have been ordered to work on other things. There has also been open talk of some Java EE developers “forking” the Java platform, breaking off with their own implementation and abandoning compatibility with the 20-year-old software platform acquired by Oracle with the takeover of Sun Microsystems six years ago. Yet Oracle remains silent about its plans for Java EE even as members of the governing body overseeing the Java standard have demanded a statement from the company.


This top-down control has caused discomfort among the previously collegial Java community. First came the 2010 departure of James Gosling, the creator of Java. “Just about anything I could say that would be accurate and honest would do more harm than good,” he wrote about his departure. He would later tell eWeek’s Daryl Taft that Oracle’s micromanagement of the Java team had stripped them of any decision-making power. Gosling’s new role amounted to being trotted out at events like a retired ball player at a sports convention.

Update (2016-07-02): Landon Fuller:

Java EE is a distinct beast, and to put it politely, not highly esteemed. Deprecating that design avenue seems wise to me.

Jeremy W. Sherman:

I wonder how much “get me the heck away from Oracle and Java” was an impetus behind IBM’s embrace of serverside Swift on Linux.

Update (2016-07-08): Jeff G (via Hacker News):

Today, a large part of my business is migrating companies off the monolithic Java EE containers into lightweight modular containers. Yes, even the tried and true banking and financial industries are moving away from Java EE. Every company has a different set of specifications for what parts they may use in a Java EE stack, but rest assured, its all going by their choice and being as small of a footprint as possible. When companies buy cloud images and the pricing is based on memory size and numbers of cores, it gets real easy to understand why they don’t want to run these stacks that require lots of resources for only needing to use less than 10% of what a Java EE stack may offer. It because it costs more to run crap that you will never use.

Oracle and I have always had a love-hate relationship over the years, primarily because they represent the quintessential corporate identity that many of us love to hate. That, along with my Apache open source roots, Oracle and I have no historical love for each other. That said, I have to somewhat defend Oracle on this decision. Many of us see this sea change. I am sure Oracle sees it as well. Oracle isn’t stupid. They see the writing on the wall.

Interviews: Kare, Bezos, Musk, Felleisen, Gruber

Susan Kare:

Susan discusses her history with Apple and icon design, and sits down with John Gruber to talk design.

Jeff Bezos (via John Gruber):

At the 2016 Code Conference, Amazon CEO Jeff Bezos talked with The Verge’s Walt Mossberg about the “gigantic” potential of artificial intelligence to change everything from shopping to self-driving cars.

There are also bits about Amazon’s retail strategy and how it thinks about Apple TV.

Elon Musk:

Tesla and SpaceX CEO Elon Musk spoke with Recode’s Kara Swisher and The Verge’s Walt Mossberg at the 2016 Code Conference about his plans to send a one-way rocket to Mars in 2018.


Musk also talked about the proliferation of electric vehicle initiatives that compete with his other company, Tesla, and why autonomous cars will become the norm. He said he doesn’t see Google as a competitor, but that “Apple will be more direct.”

Matthias Felleisen (via Shriram Krishnamurthi):

We talk his history with Schemes from The Little Lisper, How to Design Programs, to Typed Racket. We also cover teaching math to middle schoolers with Bootstrap, and using programming to teach problem solving and more.

John Gruber:

“My audience is just me,” Gruber said of his Daring Fireball site, on the latest episode of Recode Media with Peter Kafka. “It’s somebody out there who’s exactly like me and isn’t writing Daring Fireball.”


On the new Recode Media, Gruber also discussed why he doesn’t want to work for a big media company; why he looks down on Apple rumor sites and other media companies that chase pageviews; and how podcasting has come to represent about half of Daring Fireball’s revenue. He also denied the claim that Apple’s top brass has much of a say in what he writes.

Friday, July 1, 2016

Xcode Clobbered by socketfilterfw at Launch


Every time I open Xcode, there is massive hard drive churn. iStat seems to indicate that the culprit is socketfilterfw. The computer is not properly useable again for upwards of 2 minutes since the disk churn slows everything to a crawl.

I’ve been getting this, too, except that (with an SSD) the main symptom is extremely high CPU usage. It takes more like 10 minutes to resolve.

When Xcode launches, the OS X Firewall needs to scan the entire Xcode application bundle for some reason (some kind of checksum validation?). socketfilterfw is the Firewall’s agent for doing this.

It was suggested that we fiddle with the Firewall settings under the Security preference pane to try to fix this, and I’ve found that by removing the listing for Xcode and turning off the setting “Automatically allow signed software to receive incoming connections”, it seems to have stopped socketfilterfw from taking over my computer for a couple minutes at every launch of Xcode. I also tried changing Xcode’s Firewall listing from “allow” to “block”, and this did not make a difference; only removing it from the list did.

Simply removing Xcode from the list fixed the problem for me. I’m not sure what functionality I’m losing by not allowing Xcode to accept connections.

See also: Ask Different.

Spotify Update Rejected From the App Store

Peter Kafka (via Peter Steinberger):

“This latest episode raises serious concerns under both U.S. and EU competition law,” Gutierrez wrote. “It continues a troubling pattern of behavior by Apple to exclude and diminish the competitiveness of Spotify on iOS and as a rival to Apple Music, particularly when seen against the backdrop of Apple’s previous anticompetitive conduct aimed at Spotify … we cannot stand by as Apple uses the App Store approval process as a weapon to harm competitors.”


Apple doesn’t require subscription services to use its iTunes billing service, but it doesn’t allow them to use an alternate payment system within the app, as Google does. Apple charges a monthly fee of up to 30 percent for those that do use its billing system — and it doesn’t want app makers to use the apps to promote alternate subscription options outside the apps.


In Spotify’s case, the company has used Apple’s billing system for years, but passed on Apple’s fee to customers by charging $13 a month instead of the $10 a month the service sells for outside Apple’s store.

It sounds like Apple is not singling out Spotify for special scrutiny. Rather, Spotify was trying to get around Apple’s longstanding in-app purchase rules. That said, I believe said rules are anti-competitive and bad for both customers and developers.

See also: Hacker News, John Gruber, Nick Heer, Juli Clover, Andrew Cunningham, and Emily Blake.

Update (2016-07-02): John Paczkowski:

In a letter sent to Spotify general counsel Horacio Gutierrez on Friday, Apple general counsel Bruce Sewell rebutted the streaming music service’s June 26 allegations that Apple is “causing grave harm to Spotify and its customers” by rejecting a recent update to Spotify’s iOS app. “We find it troubling that you are asking for exemptions to the rules we apply to all developers and are publicly resorting to rumors and half-truths about our service,” Sewell wrote in a letter obtained by BuzzFeed News.

Juli Clover:

Sewell’s letter to Spotify ends with some clarification on why Spotify’s app was rejected on May 26. Spotify replaced its in-app subscription purchase options with an account sign-up feature Apple says was "clearly intended to circumvent Apple’s in-app purchase rules."

Why Swift Ditched Tuple-to-Tuple Function Modeling

Jordan Rose (via Jaden Geller):

The real blow, however, came with that very first idea: that we could treat methods with different argument labels as simple overloads in type. This led to poor diagnostics where the compiler couldn’t decide whether to believe the types or the argument labels, and might tell you you have the wrong argument labels rather than a type mismatch. For pretty much every Apple API, this was the wrong decision. On top of all that, it was really hard to refer to a method when you didn’t want to call it. (Most methods with the same base name still have unique labels, so you don’t need the types to disambiguate.)

So we introduced the notion of “full names”, which are the things you see written as move(from:to:) (and which are represented by DeclName in the compiler). Almost immediately diagnostics got better, testing optional protocol requirements got shorter, and a lot of compiler implementation got simpler.


And then we kind of got stuck here. We have full names used throughout the compiler, but tuple labels still appear in types. They’re still used in mangling.

Objective-C Class Properties

Andrew Madsen:

Xcode 8 ships with a new versions of Clang and LLVM (800., and 8.0.0 as of the first beta). The new version of Clang adds support for class properties in Objective-C. The feature is not yet fully documented anywhere that I can find, so I did a little experimenting.


It is important to note that unlike regular instance properties, Objective-C class properties can not be synthesized using @synthesize, nor are they automatically synthesized by the compiler. You are responsible for implementing accessor methods for them yourself. (You can also use an @dynamic for them to tell the compiler that accessor methods will be provided at runtime.)

Rest in Motion

Nate Soares (via Alexis Gallagher):

This is a failure mode that I used to fall into pretty regularly. I would model my work as a finite stream of tasks that needed doing. I’d think “once I’ve done the laundry and bought new shoes and finished the grocery shopping and fixed the bugs in my code and finished the big refactor, everything will be in order, and I’ll be able to rest.” And in that state of mind, every new email that hit my inbox, every new bug discovered in my code, every tool of mine that wore down and needed repair, would deal me damage.

I was modeling my work as finite, with the rest state being the state where all tasks were completed, and so every new task would push me further from that precious rest state and wear me down.


Look at all the streams you want to move through, assess how much bandwidth you have available, and then simply move through the streams at the appropriate clip.