Archive for June 2016

Thursday, June 30, 2016

“I’d Like You to Police Each Other”

Natalie Kerris (former Apple PR executive):

In honor of today’s 9th anniversary of the original iPhone launch: Steve Jobs PISSED OFF moments (1997-2010).

Jason Snell:

Features that classic moment when he asked journalists to stop doing their job. Sorry, Steve, no chance.

The reason there were 500 base stations is that Apple didn’t want to provide Internet for the room. That changed fast.

Last night in the @atpfm chatroom the Tipster said the reason iOS has Ethernet support is because of that demo fail.

See also: Steve Jobs FUNNIEST moments, which I like even better, and Steve Jobs contradicting moments.

Why Google Stores Billions of Lines of Code in a Single Repository

Rachel Potvin and Josh Levenberg:

The Google codebase includes approximately one billion files and has a history of approximately 35 million commits spanning Google’s entire 18-year existence. The repository contains 86TB of data, including approximately two billion lines of code in nine million unique source files.

[…]

Managing this scale of repository and activity on it has been an ongoing challenge for Google. Despite several years of experimentation, Google was not able to find a commercially available or open source version-control system to support such scale in a single repository. The Google proprietary system that was built to store, version, and vend this codebase is code-named Piper.

[…]

Most developers access Piper through a system called Clients in the Cloud, or CitC, which consists of a cloud-based storage backend and a Linux-only FUSE file system. Developers see their workspaces as directories in the file system, including their changes overlaid on top of the full Piper repository. CitC supports code browsing and normal Unix tools with no need to clone or sync state locally. Developers can browse and edit files anywhere across the Piper repository, and only modified files are stored in their workspace. This structure means CitC workspaces typically consume only a small amount of storage (an average workspace has fewer than 10 files) while presenting a seamless view of the entire Piper codebase to the developer.

[…]

Trunk-based development is beneficial in part because it avoids the painful merges that often occur when it is time to reconcile long-lived branches. Development on branches is unusual and not well supported at Google, though branches are typically used for releases. Release branches are cut from a specific revision of the repository. Bug fixes and enhancements that must be added to a release are typically developed on mainline, then cherry-picked into the release branch. […] When new features are developed, both new and old code paths commonly exist simultaneously, controlled through the use of conditional flags. This technique avoids the need for a development branch and makes it easy to turn on and off features through configuration updates rather than full binary releases.

The Android and Chrome teams are separate and use Git.

Seeing Apple Through Color Blind Eyes

Jason Snell:

There are a few video games that I’m unable to play because they require quick sorting of colors that I have difficulty differentiating. Some of them have special modes for people like me; others just never think about it. The developers of Destiny did a lot of work to update the game’s controls and heads-up display in order to make it more usable by people who can’t perceive colors the way most people do.

[…]

I can’t really tell the difference between the green and the amber lights at a glance. For a long time, I didn’t even realize the light on the MagSafe connector changed color. (Apple is far from alone here — lots of electronics use subtle color shifts of an LED to indicate things in a way that I simply can’t digest.)

Summer Travel Tech Hacks

David Pogue:

In this video, I’ve distilled all my tricks for painless, speedy plane travel into a brisk, illustrated three minutes. For your reference, here are the sites and apps I’ve come to rely on[…]

Wednesday, June 29, 2016

Safari 10 Brings Native App Extensions

Daniel Dilger (via Dave Mark):

Like previous App Extensions, the new architecture defines a broad Extension Point for Safari that allows third party developers to add new functionality to Safari, both to read and modify web page content (such as translating text into another language) and to communicate back and forth with a native app to integrate app data into Safari or to get web data into an app.

Developers can extend the Safari user interface by adding a toolbar button to execute a command or display a popover window, add a contextual menu item, inject a style sheet that alters how web pages are presented (such as modifying fonts used or text sizes), or inject JavaScript that changes how a page behaves or enables it to communicate with the app extension.

[…]

More importantly, the new App Extensions architecture enables developers to distribute Safari Extensions as part of their app through the App Store.

Safari App Extension Programming Guide:

Safari app extensions are available in OS X 10.12 and later and in OS X 10.11.5 when Safari 10 is installed. Using a Safari app extension, you can add new functionality to Safari, read and modify web page content, and communicate with your native application to integrate its content into Safari or send web data to your app.

Safari app extensions are written using a combination of JavaScript, CSS, and native code written in Objective-C or Swift.

Input Masks: Violating User Expectations

Lukas Mathis:

Because there are no standardized, widely accepted behaviors for input masks, it’s best to avoid «magical» behaviour (e.g. automatically entering characters that the user did not type), or, if you do need magical behavior, also account for user behavior that does not expect magical behavior.

For example, if the field auto-tabs to the next field, and the user also tabs right after filling in a field, it might be best to ignore the user’s tab.

[…]

If you create a new control that kind of looks like an existing control, the new control should not violate people’s expected behavior of the existing control. In other words, text fields with input masks’ behavior should be as close to a normal text field’s behavior as possible.

A Year of Windows Kernel Font Fuzzing

Mateusz Jurczyk:

To most readers of this blog, the fact that fonts are a very significant attack vector does not have to be reiterated. There are a number of different file formats in active use. These formats are extremely complex, both structurally and semantically. As a result, they are correspondingly difficult to implement correctly, which is further amplified by the fact that a majority of currently used font rasterizers date back to (early) 90’s, and were written in native languages such as C or C++. Controlled font files are also deliverable through a variety of remote channels – documents, websites, spool files etc. Last but not least, the two powerful virtual machines executing programs describing glyph outlines in the TrueType and OpenType formats have proven vastly useful for creating reliable exploitation chains, thanks to the ability to perform arbitrary arithmetic, bitwise and other operations on data in memory. For all of these reasons, fonts have been an attractive source of memory corruption bugs.

[…]

If nothing else, the effort and its results are evidence that fuzzing, if done correctly, is still a very effective approach to vulnerability hunting, even with theoretically “mature” and heavily tested code bases. Furthermore, the two bug collisions prove that Windows kernel font bugs are still alive and kicking, or at least were actively used in the wild in 2015. In the second post of the series, we will discuss the meaty parts of the research: how we prepared the input corpus, mutated and generated interesting font samples, fuzzed the Windows kernel at scale, reproduced the crashes and minimized them.

Vintage Macworld Magazine Library

VintageApple (via Clark Goble):

There is no better depiction of the history of the Macintosh than the old issues of MacWorld Magazine.

For your reading pleasure is a complete MacWorld Magazine collection from 1984 through 1994.

Color scans in PDF format. I really enjoyed flipping through some old issues of Macworld (note the capitalization) recently before recycling them. The ads, in particular, are very different from what we’re used to seeing now.

Tuesday, June 28, 2016

Behind Apple’s Advanced Computer Vision for Photos.app

Kay Yin (via Accidental Tech Podcast):

Photos app supports detecting 4,432 different scenes and objects. These scenes or objects can be searched for in all languages.

Additionally, you can search for various landmarks. For example, Photos can respond for search query of “Maho” (beach in Saint Martin), despite Photos is not programmed or trained to understand specific landmarks. Behind the scenes, Photos app first generates a generic categorization for the scene, “beach”, then searches through a built-in dictionary for all landmarks that has the name “beach” in its definition. Therefore, cleverly, despite Photos app knows nothing about “Maho” in particular, it is still able to return the right results. The same applies to nature scenes, water scenes and urban scenes.

For your pleasure, here’s a comprehensive list of scenes you can search for.

I would love to see this kind of thing in Lightroom.

Scrivener Syncing

Keith Blount:

It’s not just the format itself, but how it is used. Because Scrivener projects comprise many files, it’s important that any given project is synced in its entirety and not piecemeal. If an updated binder structure is downloaded, it’s not much use if none of the new files it references are yet available, for instance. Dropbox gives us the control we need in this regard.

This is also the reason that the sync needs to be invoked manually rather than continually happening in the background. With background sync (which iCloud always uses), there is no control over which files appear when, meaning that important structural files could download without the files they refer to or vice versa. This is an even more serious problem when the connection is lost part-way through a sync. All too easily this could put a project into an inconsistent state that could wreak havoc, and we’d rather have users complaining that they don’t like manual sync than that they have just lost four hours of writing or structural changes!

Open Source TaskPaper Model Layer

Birch Outline (via Jesse Grosjean):

Cross-platform scripting for TaskPaper.

  1. Model: Outline contains Items each of which has attributes and an AttributedString body.
  2. Runtime: Change events, undo support, query evaluator, and relative date/time parsing.
  3. Serialization: Read/Write .taskpaper, .opml, and .bml

Suitable for processing TaskPaper files wherever there's JavaScript. Also potentially useful when creating your own apps that need an outline model.

There’s also a Swift wrapper.

Removing the iPhone’s Headphone Jack

Nilay Patel:

Another day, another rumor that Apple is going to ditch the headphone jack on the next iPhone in favor of sending out audio over Lightning. Or another phone beats Apple to the punch by ditching the headphone jack in favor of passing out audio over USB-C. What exciting times for phones! We’re so out of ideas that actively making them shittier and more user-hostile is the only innovation left.

Look, I know you’re going to tell me that the traditional TRS headphone jack is a billion years old and prone to failure and that life is about progress and whatever else you need to repeat deliriously into your bed of old HTC extUSB dongles and insane magnetic Palm adapters to sleep at night. But just face facts: ditching the headphone jack on phones makes them worse, in extremely obvious ways. Let’s count them!

John Gruber:

Should the analog headphone jack remain on our devices forever? If you think so, you can stop reading. If not, when? Maybe now is the wrong time, and Apple is making a mistake. I don’t know. None of us outside the company seem to know, because all that has leaked is that the new iPhone won’t have the port, with no explanation why. But I say at some point it will go away, and now seems like it might be the right time. Also, historically, Apple has proven to be very good at timing the removal of established legacy ports.

[…]

If it weren’t for Apple we’d probably still be using computers with VGA and serial ports. The essence of Apple is that they make design decisions “no one asked for”.

[…]

And in five years we’ll look at analog headphone jacks the way we look at all the other legacy ports we’ve abandoned.

Steve Streza:

As if Apple should make a change just because this port is old. No, the right question is not “when”, it’s this: Is the proposed solution better enough than the status quo to justify a disruptive transition?

We don’t know how Apple will position it as superior, as John acknowledges right here. We’re just supposed to assume that this is a better solution, and that it “seems like it might be the right time”, based on… who knows. Until Apple answers that question, which they yet haven’t done publicly or through controlled media leaks, we can only speculate.

[…]

So barring some bombshell announcement that changes how we think about digital headphones, this is a step forward and 7 steps back.

[…]

It is nigh impossible in mobile for existing customers to “just vote with your wallet” any more without significant time and cost.

What is good for Apple is not inherently good for you. There are times when what Apple does is in sync with what benefits their customers, such as in areas of privacy and security. But that same company released Apple Maps when it was literally years away from being ready for the real world, because it benefitted them to not deal with Google. The same company that forced U2 into everyone’s iTunes libraries without consent as a marketing ploy. The same company that made their music app frustratingly difficult to use so they could shove their subscription service into every nook and cranny of the user interface. Apple is not on your side by default.

John Gruber:

I didn’t argue that this change will be good for users. I argued that it could be. We don’t know yet!

John Gruber:

One of the ideas I’ve seen bandied about regarding the purported removal of the standard headphone jack on this year’s new iPhones is that maybe it’s because the iPhone is switching from Lightning to USB-C. […] I say no way.

John Gruber:

What if Apple is planning on Bluetooth earbuds that include a Lightning jack, like the Pencil? Plug them in to the device you want to pair them with, tap “Pair”, and you’re done. Easy to charge, too. (But again, this only works across iOS and Mac if Macs gain Lightning ports.)

See also: Marco Arment on The Talk Show.

The bottom line for me is that no one seems to have any idea what the benefits would be to justify the hassle. The TRS connector is not as obviously antiquated as the floppy drive was. And Lightning and Bluetooth headphones are possible without removing it. I most often use the headphone jack in the car or on a plane when I’m simultaneously charging the phone via the Lightning port. It sounds like doing this with the iPhone 7 would require dongles, if it’s possible at all.

Sidenote: I think the real problem with the original iMac was not the lack of a floppy drive—you could easily add an external one if needed—but the removal of high-speed I/O. The CD drive was not writable, and flash drives were not yet plentiful. USB 1.1 was an improvement over serial ports, but it was terrible for connecting external storage. SCSI was removed and replaced with nothing until the iMac DV added FireWire.

Update (2016-06-30): Nilay Patel and Frank Bi (via John Gruber):

But how long does it really take Apple to kill legacy tech? We threw together a chart to map it out.

Update (2016-07-01): Jason Snell:

I’ve heard a lot of reasons why people think Apple will remove the headphone jack, but as I said this week on Upgrade, I’m not sure I’ve heard a single one that makes sense or justifies the inconvenience that would be the result of such a move. Let’s go through them.

[…]

What is this, “Logan’s Run”? The age of something isn’t reason enough to kill it.

Marco Arment:

Apple better have very good benefits for this that customers will want, but none of the reports so far indicate any.

Combined with the disappointment sure to result from the same physical iPhone design for three years in a row — a mediocre one, at that — I fear for the public perception of this fall’s iPhone and Apple as a result.

Eric Blair:

Without AppleCare, this headphone jack represents a $180 increase in repair costs.

Update (2016-07-02): See also: The Talk Show with Nilay Patel.

Update (2016-09-06): Jesper:

I am dumb enough for many things, but not to believe that this will prevent Apple from going this route. They will set this path subject only to their own discretion, and I praise their freedom to do so. What I can’t quite get over is seeing people who were open earlier to seeing the alternate solution, the unseen parts of this equation, what only Apple knows right now, are taking delight already in consigning anyone not already dancing in the streets over the loss of the shackles of old jacks to the slow luddite moron basket. So much for sticking to the argument, seeing it play out, and possibly even taking Apple to task for killing something that was actually simplicity in its purest form over something that – for the purposes of being a replacement – is simple in wolf’s clothing, bringing side effects, needless manuevering, repetetive loss of connection and general anguish in its wake.

[…]

My only prediction for Wednesday’s event is that there will be no “it just works” over the wireless headphone story for iPhone 7.

Ben Lovejoy:

All of which makes me strongly suspect that Apple planned all along to deliberately leak the fact that it would be dropping the 3.5mm socket – even if Makotakara discovered it first. It did so, I believe, because it knew that there would be outrage from some quarters, most notably those who own expensive wired headphones which they like to use with the iPhones as well as other kit.

Leaking the news such a long way in advance would ensure that people had time to prepare, both emotionally and practically.

David Pogue:

The headphone jack is what’s preventing phones from getting any thinner. It’s the limiting factor.

A lot of people really love thin phones. But if you don’t care about thinness, you can put it another way: The headphone jack is what’s preventing phone batteries from getting bigger. You do care about battery life, right?

[…]

In the post-headphone jack era, your music will remain digital until it reaches the headphones, which can have a much nicer converter. You’ll skip over that analog conversion business—and get better-sounding audio.

Monday, June 27, 2016

Thoughts on the New Official Apple App Icon Template

Michael Flarup (via Nick Heer):

I think it’s great that Apple wants to provide templates for the design resources they’re asking developers to deliver, but the ones they’ve put out is not only a lot less practical than the 3rd party resources available, they’re also factually wrong — using bitmap roundrects instead of the platform standard squircle. Until Apple feels like dedicating more time to helping developers deliver better assets, it looks like there’ll be plenty of reasons why we still have 3rd party resources around.

AppleScript “Standard Libraries” Project

Hamish Sanderson:

As you may recall, back at the start of this year I was bitching about the [AppleScript] team adding a native library loader to AppleScript in 10.9 yet bizarrely forgetting to provide any general-purpose libraries along with it, after which I decided to go write the damn things myself just so AS users would at least have something decent to use. The result was a set of seven ‘standard libraries’ (plus one support and one code testing library) providing commands for a wide range of everyday tasks[…]

[…]

Anyway, long story short, I got it about 98% done before running out of both time and motivation - the implementation is feature-complete and generally pretty polished and bombproof, but there’s still some holes to fill in test coverage and user documentation.

[…]

So, if the AppleScript community would like a chance to help itself then here it is: I want someone experienced in AppleScript and [AppleScriptObjC], and preferably with some actual programming skills as well (cos this is not just toy code), to step up and officially take the whole project over from me over the next couple of weeks. That’s full ownership, BTW: all copyright, choice of license, project management, etc.

Here’s the GitHub project.

Homescreen Sharing App Rejected From App Store

Préshit Deorukhkar:

Apple executives explained to us that we cannot showcase a homescreen springboard image within the app — stating that the springboard was Apple’s IP and it was against Apple’s Brand guidelines.

The rejection was not only disappointing and heart-breaking for us, as we had put several months of hard work into the app, but we also found the reason very hard to believe.

[…]

Homescreen.me is merely a platform that showcases user submitted content, something that other apps like Twitter, Facebook, Reddit etc. already do. Example — what if someone shares a homescreen image on Twitter? Is that a violation too?

Unfortunately, Apple just wouldn’t budge. A work-around, we were told, was to offer users a simple text area where they can type which apps they use on their Homescreens.

Update (2016-06-27): Nick Heer:

Betaworks launched a similar app called “#Homescreen” which, though it had app review issues of its own, was never sanctioned for intellectual property issues.

Update (2016-06-29): Hwee-Boon Yar:

I built an app that simulates homescreen for testing icons. Similar problem.

App Store and iTunes Store Outage

Juli Clover:

Some iOS and Mac users are currently unable to access the App Store, Mac App Store, and iTunes Store, according to reports from MacRumors readers and Twitter users.

[…]

Apple’s system status page has not yet been updated to reflect any kind of outage and it is not clear how many users are affected.

Greg Barbosa:

Today’s outage comes a few weeks after a previous widespread outage took out Apple Music, iCloud Music Library, and iCloud. Apple has previously remained quiet when these outages occur, relying on using the System Status page to communicate to customers of any issues. Zac Hall had previously delved into how Apple should better handle these situations, especially when they expect customers to put more faith in them for future cloud-based services.

Luc Vandal:

Yesterday’s App Store sale numbers. Ouch! #outage

Previously: App Store Search Is Currently Broken.

Update (2018-06-18): Benjamin Mayo (Hacker News):

Apple Maps has been down for many hours today, causing mayhem for customers seemingly worldwide. Whilst Apple’s system status reports no issues, users across Twitter and Apple Support forums are finding that is currently impossible to search for locations or get directions. The map tiles appear to be loading normally, but points of interest and critical functions of the app are simply not working.

Friday, June 24, 2016

UserNotifications.framework

Michał Kałużny:

Obviously, Apple didn’t go to all that trouble only because they wanted us to use a different (and obviously better) API. The UserNotifications framework actually gives us access to many new, long-awaited features.

Update (2016-06-27): Nick Heer:

Unfortunately, these enhancements haven’t made their way into MacOS.

Feet in Both Worlds: From Objective-C to Swift

Andy Matuschak:

This morning we’re gonna talk about Objective-C and Swift interop. We’re gonna talk about it really from two perspectives. We’re gonna talk about it from a theoretical perspective, that of language design. […] And then we’re gonna talk about what it was like for us to take a […] 20,000 to 30,000 line Objective-C app, and start adding 20,000 to 30,000 lines of Swift to it, and to add those lines in a very Swifty style.

Interoperability makes it possible to integrate [Swift] features back into Objective-C code with no hassle […] without having to rewrite the entire app in Swift at once.

This is a quote from the Swift book, this is a lie from the Swift book. It’s a nice lie. I mean, it’s kind of true. It’s kind of true if you write bad Swift. But then, it’s basically just syntax sugar versus Objective-C, which is nice, but which is maybe not worth the complexity. […] Okay, so we had this plan, because the Swift book told us this thing, and we believed it. And we embarked about 13 months ago on a large project, we being Khan Academy, to write a large product in Swift, large existing code based, running a large new product.

It was not great, it was emphatically not great. So along the way, we were constantly forced to rewrite large swathes of our app in Swift as we went, or we were forced to decide not to use Swifty idioms and semantics, which is not a decision that we really enjoyed having to make. I think it’s fine. I’m not saying, “Oh, you shouldn’t use Swift in existing apps.” But what I am saying is, “You should add a lot of padding to your estimates.” ‘Cause it slowed everything down a lot, for months. Many, many, many months. And so there’s people out there who are just saying “Well, screw it. I’m just gonna rewrite the whole thing in Swift.” I don’t know, that might have been a better strategy. Transliterating Objective-C to Swift is pretty easy.

This is a conference talk from November 2015, before Swift 2.2 and Swift 3, but a lot of the issues still apply.

The Dissertation Behind rsync

Andrew Tridgell (PDF, via @CompSciFact):

This thesis presents efficient algorithms for internal and external parallel sorting and remote data update. The sorting algorithms approach the problem by concentrating first on highly efficient but incorrect algorithms followed by a cleanup phase that completes the sort. The remote data update algorithm, rsync, operates by exchanging block signature information followed by a simple hash search algorithm for block matching at arbitrary byte boundaries. The last chapter of the thesis examines a number of related algorithms for text compression, differencing and incremental backup.

Modern Core Data With Swift

Daniel Eggert:

We have used protocol and protocol extensions to make our code more readable.

We have also used a few other tricks, but the main gist is that you can create small helpers in your code to make your life easier, and make life easier for other people that have to read your code. Using old APIs is great because they have been battle-tested; it is code that has been around for many years… and, for many years, bugs have been fixed. But we can make these APIs even better (and make our lives easier).

When we create these helpers, it is important to keep the spirit of the original implementation. We want to make it easy for other people who read the code, and that who may not know about these wrappers.

Thursday, June 23, 2016

Restructuring Swift Condition Clauses

SE-0099 (via Erica Sadun):

Swift condition clauses appear in guard, if, and while statements. This proposal re-architects the condition grammar to enable an arbitrary mix of Boolean expressions, let conditions (which test and unwrap optionals), general case clauses for arbitrary pattern matching, and availability tests. It removes where clauses from optional binding conditions and case conditions, and eliminates gramatical ambiguity by using commas for separation between clauses instead of using them both to separate clauses and terms within each clause. These modifications streamline Swift’s syntax and alleviate the situation where many Swift developers don’t know they can use arbitrary Boolean conditions after a value binding.

Bravo.

Swift Protocols Constrained to a Specific Class

BJ Homer:

Objective-C developers learning Swift often wonder how to represent something like UIViewController<MyProtocol>. The simple answer is that there's currently no way to directly represent that construct in Swift as of Swift 2.2. However, it's possible to work around it fairly easily with just a tiny bit of extra code.

What is Differential Privacy?

Andy Greenberg (Hacker News, MacRumors):

“We believe you should have great features and great privacy,” Federighi told the developer crowd. “Differential privacy is a research topic in the areas of statistics and data analytics that uses hashing, subsampling and noise injection to enable…crowdsourced learning while keeping the data of individual users completely private. Apple has been doing some super-important work in this area to enable differential privacy to be deployed at scale.”

Differential privacy, translated from Apple-speak, is the statistical science of trying to learn as much as possible about a group while learning as little as possible about any individual in it. With differential privacy, Apple can collect and store its users’ data in a format that lets it glean useful notions about what people do, say, like and want. But it can’t extract anything about a single, specific one of those people that might represent a privacy violation. And neither, in theory, could hackers or intelligence agencies.

Matthew Green (Hacker News):

To make a long story short, it sounds like Apple is going to be collecting a lot more data from your phone. They’re mainly doing this to make their services better, not to collect individual users’ usage habits. To guarantee this, Apple intends to apply sophisticated statistical techniques to ensure that this aggregate data -- the statistical functions it computes over all your information -- don’t leak your individual contributions. In principle this sounds pretty good. But of course, the devil is always in the details.

[…]

The total allowed leakage is often referred to as a “privacy budget”, and it determines how many queries will be allowed (and how accurate the results will be). The basic lesson of DP is that the devil is in the budget. Set it too high, and you leak your sensitive data. Set it too low, and the answers you get might not be particularly useful.

[…]

A much more promising approach is not to collect the raw data at all. This approach was recently pioneered by Google to collect usage statistics in their Chrome browser. The system, called RAPPOR, is based on an implementation of the 50-year oldrandomized response technique.

[…]

The main challenge with randomized response systems is that they can leak data if a user answers the same question multiple times.

Bruce Schneier:

What we know about anonymization is that it’s much harder than people think, and it’s likely that this technique will be full of privacy vulnerabilities. (See, for example, the excellent work of Latanya Sweeney.) As expected, security experts are skeptical.

Theoretical Anatagonist:

Microsoft Research the birth place of Differential Privacy has abandoned it.

[…]

In order to asses accuracy/utility/privacy trade-off correctly one must posses enough information about queries expected, and how much privacy budget should be allocated towards them. Keeping track of queries and budget consumed is even more difficult. Certain tasks require answers to be computed as exactly as possible. The impact from answering such queries and resulting depletion of privacy budget has never been examined in literature.

[…]

There have been few attempts at rigorously evaluating privacy budgets and parameters such as epsilon. However they have only shown Differential Privacy to be impractical.

[…]

You will encounter strong resistance and in some cases outright bullying from vocal supporters of Differential Privacy. These people will likely be completely unfamiliar with your area of application and existing practices. They will barge in into public meetings proclaiming that Differential Privacy is the only real solution. And that researchers who have spent years working with data in that field should be ignored. Even when several papers have called for further study of Differential Privacy before adoption in practice.

Rutgers:

Moritz Hardt presents a “Tutorial on Differential Privacy in the Theory Community” at the DIMACS Workshop on Recent Work on Differential Privacy Across Computer Science, held Wednesday, October 24th 2012 at the Computing Research & Education Building on Busch Campus of Rutgers University. This event was presented under the auspices on the DIMACS Special Focus on Information Sharing and Dynamic Analysis and the DIMACS Special Focus on Cybersecurity.

Update (2016-06-27): Juli Clover:

There’s been a lot of confusion about differential privacy and what it means for end users, leading Recode to write a piece that clarifies many of the details of differential privacy.

First and foremost, as with all of Apple’s data collection, there is an option to opt out of sharing data with the company. Differential data collection is entirely opt in and users can decide whether or not to send data to Apple.

Wednesday, June 22, 2016

Xcode 8 Tips and Issues

Sash Zats:

Amazing tip from Xcode labs: add a EXCLUDED_SOURCE_FILE_NAMES build setting to conditionally exclude sources for different configuration.

Dave DeLong:

There’s also an INCLUDED_SOURCE_FILE_NAMES, and they can be used together (exclude that, but include this).

Daniel Jalkut:

A major change from Xcode 7 is that Xcode now requires that every target in your project specify a development team, even when the manual code signing option is selected.

[…]

The new PROVISIONING_PROFILE_SPECIFIER build setting will impart the desired team information to Xcode, even if provisioning profiles are not used.

[…]

This doesn’t exactly solve the challenge for open source projects, because organizations using an open source project will still have to override the configuration file or build settings for the projects they check out.

Daniel Jalkut:

So, the fix in Xcode 8, with the 10.12 SDK, is to delete those parentheses so Swift appreciates that it’s accessing a property. But doing so will break the build back on the 10.11 SDK. What’s a forward-looking, backward-looking developer to do?

[…]

Unfortunately, I think the only way to convince Swift to absolutely ignore the problematic line is to use a Swift compiler check, instead. Note that this is a bad solution because we are fundamentally not concerned about the version of Swift being used.

[…]

In my opinion, we need a compile time, conditional code exclusion that operates on the SDK, similarly to #available. Something like “#if sdk(OSX 10.12)”. As far as I know, nothing like this exists.

Russ Bishop:

Source Extensions have several capabilities and are organized around commands. Each extension can provide a list of commands in its plist file or via an override of XCSourceEditorExtension. Each command specifies what XCSourceEditorCommand subclass implements the command, the name on the menu, and a unique identifier. The same class can implement several commands by switching on the identifier.

[…]

The first parameter is an XCSourceEditorCommandInvocation. That invocation has the contents of the editor’s text buffer (either as a single string or as an array of lines). It also has the selections, and each selection has a start and end that tell you the line and column (which can be used to index into the text buffer’s lines array). If nothing is selected, the single XCSourceTextRange in the array will have the same start and end point representing the insertion point.

See also: WWDC Session 414: Using and Extending the Xcode Source Editor.

Update (2016-06-23): Jay Graves:

Xcode 8 brings the dark art of customized code signing and provisioning profiles into the light for all to see. Instead of being buried in the Build Settings, Code Signing is now displayed front and center in the General tab. When “Automatically manage signing” is unchecked we can specifically select the provisioning profile we wish to use for each Configuration. Now this in itself is nothing new, we have always been able to do this in the Build Settings tab. The awesome sauce comes in with the smarts that Xcode 8 displays here. The drop down list of profiles now groups the Eligible profiles above the Ineligible profiles. Meaning that profiles that match the bundle identifier absolutely or with a wild card will be displayed above any profiles that don’t match at all.

[…]

The Xcode team has included a new report in the Report Navigator that outlines everything Xcode does on your behalf related to code signing.

Update (2016-06-29): James Thomson:

Xcode 8 no longer supports iOS 7 as a deployment target, so if you add support for iOS 10, you’ll lose iOS 7 support.

Matt Gallagher:

I took a break from converting projects to Swift 3 this weekend and instead played around with writing a new “Source Editor Command” extension for Xcode 8. The result is an extension that detects and corrects whitespace issues. It’s a topic I’ve touched on before because I’m frustrated by the continuous burden of correcting whitespace in my projects. I wanted to see if an Xcode source extension could reduce my annoyance with this issue.

The fun part of this exercise for me was writing a pushdown automaton parser where all of the parser logic was expressed through the case patterns of a single Swift switch statement. Accordingly, most of this article is not really about Xcode Source Editor Command extensions or whitespace issues but is actually about design choices related to large, complex switch statements in Swift: choosing data types to switch upon, designing case patterns to handle a deep decision tree in a single switch statement, when to use custom pattern matching and when to use where clauses.

Update (2016-10-14): Russell Ivanovic:

To stop the Xcode 8 iOS Simulator from logging like crazy, set OS_ACTIVITY_MODE = disable in your debug scheme.

Script Debugger 6

Mark Alldritt:

I’m pleased to announce the release of Script Debugger 6. This release introduces over 25 new features and embraces modern AppleScript, including AppleScriptObjC. To sweeten the deal, Script Debugger’s price has been reduced by 50%!

Late Night Software:

If you distribute scripts, chances are you want them to pass muster with Mac OS X’s Gatekeeper, and that means code signing them. Script Debugger 6 makes this task almost invisible with its always-on code signing. Turn it on, and your script is code signed every time it is saved. No more exporting, no more forgetting.

[…]

AppleScript’s progress properties are a great way of displaying a script’s progress in an applet, and now you can display them as you run your scripts in Script Debugger 6.

[…]

Before Script Debugger 6, working with AppleScriptObjC required specialist software. This powerful technology often requires you to type verbose code, and is unforgiving about case — something foreign to typical scripters. Script Debugger 6’s code completion tools do much of the typing for you and address common problems. Debugging AppleScriptObjC is a bear because AppleScript cannot display values you are working with. Script Debugger 6 fixes this by displaying AppleScriptObjC values as you step through your code — there is no more guessing.

Script Debugger 6 Release Notes:

Script Debugger 6 at long last provides code folding where you can collapse block structures and comments within your code to reduce clutter when editing.

[…]

Script Debugger 6 performs each script compilation in a new AppleScript compiler instance. This avoids a long standing problem where it was impossible to change the case of a variable, property, handler or script object once the script had been compiled.

Update (2016-06-29): See also: Howard Oakley.

Alan Kay AMA

Alan Kay:

I’m not a big Unix fan -- it was too late on the scene for the level of ideas that it had -- but if you take the cultural history it came from, there were several things they tried to do that were admirable -- including really having a tiny kernel and using Unix processes for all systems building (this was a very useful version of “OOP” -- you just couldn’t have small objects because of the way processes were implemented). It was quite sad to see how this pretty nice mix and match approach gradually decayed into huge loads and dependencies. Part of this was that the rather good idea of parsing non-command messages in each process -- we used this in the first Smalltalk at Parc -- became much too ad hoc because there was not a strong attempt to intertwine a real language around the message structures (this very same thing happened with http -- just think of what this could have been if anyone had been noticing …)

[…]

It used to be the case that people were admonished to “not re-invent the wheel”. We now live in an age that spends a lot of time “reinventing the flat tire!”

The flat tires come from the reinventors often not being in the same league as the original inventors. This is a symptom of a “pop culture” where identity and participation are much more important than progress…

[…]

When the first Turing Award winner -- Al Perlis -- was asked in the 60s “What is Computer Science?”, he said “It is the science of processes!”. He meant all processes including those on computers, but also in Biology, society, etc.

His idea was that computing formed a wonderful facility for making better models of pretty much everything, especially dynamic things (which everything actually is), and that it was also the kind of thing that could really be understood much better by using it to make models of itself.

[…]

I’ve been constantly surprised about how what I called “object-oriented” and “system-oriented” got neutered into Abstract Data Types, etc., (I think because people wanted to retain the old ways of programming with procedures, assignment statements, and data structures. These don’t scale well, but enormous amounts of effort have been expended to retain the old paradigms …

Tuesday, June 21, 2016

Increased Safety in Swift 3

Matt Mathias:

Things are different with Swift 3.0. The ! in init! indicates that the initialization process can fail, and if it doesn’t, that the resulting instance may be forced (i.e., implicitly unwrapped). Unlike in Swift 2.x, instances resulting from init! are Optionals and not ImplicitlyUnwrappedOptionals. That means you will have to employ optional binding or chaining to access the underlying value.

[…]

We never explicitly declared that nilPerson should be an ImplicitlyUnwrappedOptional. And it’s not great that the compiler inferred unsafe type information.

Swift 3.0 solves this problem by treating ImplicitlyUnwrappedOptionals as Optional unless we explicitly declare that we want an ImplicitlyUnwrappedOptional.

iOS 10’s Design Language

Serenity Caldwell:

Gone is the awkward mix of swiping: Instead, the Lock screen as displayed takes advantage of content in each direction; swipe from the top for Notification Center, right for the Widgets screen, left for the Camera app, or bottom for Control Center. Along the bottom of the Lock screen are page indicators like those on the Home screen — another indicator that more content hides out of reach on each side.

[…]

With these changes, iOS 10 paints a much clearer visual map for users, built in layers: On the top layer, you have Actions: Notification cards, Widgets, Control Center, and 3D Touch options; on the bottom layer, you have the Home screen; and the middle layer holds your Lock screen (if locked) or other app-related content (if unlocked).

[…]

By making each bit of iOS UI its own distinct layer, this also opens up greater horizontal interactions, as with Control Center. As seen in the keynote, Control Center now lives inside a hovering bubble layer, with a Music widget and Home widget just a left swipe away.

[…]

As part of iOS 7, Apple bet big on thinner font faces legible at small sizes; with iOS 10, the company is re-embracing font size and weight.

[…]

In part, the bigger font faces provide for readable, tappable touch targets: In the future, instead of stabbing furiously at the tiny mini-player on the bottom of the iOS 9 app, I can look forward to tapping an element double in size, with clearly demarcated play/pause buttons and no More (...) button in sight.

John Gruber:

I like these buttons that look like buttons in iOS 10.

Next Steps for Legacy Safari Plug-ins

Ricky Mondello:

By default, Safari [10] no longer tells websites that common plug-ins are installed. It does this by not including information about Flash, Java, Silverlight, and QuickTime in navigator.plugins and navigator.mimeTypes. This convinces websites with both plug-in and HTML5-based media implementations to use their HTML5 implementation.

[…]

When a website directly embeds a visible plug-in object, Safari instead presents a placeholder element with a “Click to use” button. When that’s clicked, Safari offers the user the options of activating the plug-in just one time or every time the user visits that website. Here too, the default option is to activate the plug-in only once.

Monday, June 20, 2016

End of the Road for MacNN

Charles Martin:

It’s kind of amazing we made it this far - only TidBITs and MacWorld are older and still around - but we’ve been told we’re packing it in. We’ve joked before that Apple becoming a huge mainstream company is the worst thing that ever happened to us, but it’s true: there’s less need for an Apple-specific news site when news about Apple is plastered everywhere, on every site, all the time. This is not the sole reason why we’re having to give up our comfy home (and just after repainting it, too!), but it’s part of the reality we’ve been working in.

I’ve been reading MacNN since the beginning. It was one of the essential “blogs” from the early Mac Web, along with sites like MacInTouch, MacFixIt, MacCentral, and The Mac Resource Page.

Update (2016-07-09): Adam C. Engst:

Sadly, the Mac news site MacNN, founded in 1995, has announced that it will no longer be publishing regularly, and the venerable New York City Apple reseller Tekserve is closing its doors for retail sales and service. On the upside, particularly for those trying to establish the technical particulars of some older Mac model, the Apple spec site EveryMac.com celebrated its 20th anniversary on 2 July 2016 and the spec app Mactracker marked its 15th anniversary back on 14 May 2016.

Disabling Xcode 8 Font Smoothing

Since Mac OS X 10.9, Xcode hasn’t followed the normal system preferences for controlling font smoothing. Lately, I’ve been using a plug-in to work around this. Xcode 8 has a new editor extensions feature but won’t load old-style plug-ins. Fortunately, Xcode 8 has changed so that it now respects the standard screen fonts preference, without having to use a plug-in:

defaults write com.apple.dt.Xcode NSFontDefaultScreenFontSubstitutionEnabled -bool YES

Note that whereas before, the preferences window would obey the setting but the text editor wouldn’t, now it’s the reverse. You can’t see the actual fonts as you’re choosing them, but they do work when you’re editing code. Unlike in other apps, the font smoothing in the editor seems to be either completely off or completely on. It does not respect the AppleSmoothFixedFontsSizeThreshold setting to only disable smoothing for smaller sizes.

Friday, June 17, 2016

The Deprecation of iCloud Core Data

In March 2013, I chronicled the problems developers were having with Core Data iCloud syncing. Many concluded that the basic NSPersistentStoreUbiquitousContent design was flawed. Bugs could be fixed, but it would never be able to do what people wanted. Apple didn’t give the impression that it thought there were major problems. It kept reporting minor improvements and bug fixes. When CloudKit was announced in 2014, it had no official Core Data story. As far as I know, there is still isn’t one. Meanwhile, Drew McCormack built the Ensembles framework, which fixes just about all the problems with Apple’s design and works with both CloudKit and Dropbox.

Fast forward to WWDC 2016. When installing the Xcode 8 beta, I noticed that all of the symbols related to iCloud Core Data were marked as deprecated in macOS 10.12 and iOS 10, with the comment “Please see the release notes and Core Data documentation.” Strangely, the Core Data release notes and What’s New in macOS 10.12 documents make no mention of this. What’s New in iOS 10 simply says that “Several NSPersistentStoreCoordinator symbols related to ubiquitous content” have been deprecated.

I asked on Twitter what was up, and no one seemed to know. No one had anything nice to say about iCloud Core Data, either, though no doubt many developers still rely on it. I hoped that all would be explained in today’s WWDC Session 242: What’s New in Core Data. Alas, reports are that absolutely nothing was said about iCloud in the session. A key technology that underpins apps is (apparently) to be removed, and Apple didn’t tell anyone at its own developer conference. In my view, this is shameful. The issue is more acute than with a normal deprecation because the APIs rely on cloud services, which could conceivably stop working even for older OS versions where they are not deprecated.

It seems clear that the way forward is Ensembles, unless you want to write your own sync engine or wait for Apple to announce one. I hope that the deprecation will spur more of the community to coalesce around Ensembles.

On a more positive note, it looks like there are a lot of great developments in Core Data this year, including the integration of a mogenerator-like tool into Xcode and, apparently, the end of NSObjectInaccessibleException, which doesn’t play well with Swift.

Update (2016-06-18): Drew McCormack:

If you move to Ensembles now, you not only get the usual support, documentation, and source code, you also get it all at half the standard price. The sale will continue until macOS Sierra is released later this year.

Update (2016-06-19): See the comments on Hacker News.

Update (2016-06-22): Apple’s Ben Trumbull:

Apps will continue to work for the foreseeable future. Aside from the deprecation warnings, there are no changes to or removal of the functionality in iOS 10. We don’t have any specific time line to announce now, but traditionally deprecated symbols on our platform remain functional for a considerable period of time before removal.

Only the client Core Data iCloud API symbols are deprecated. Core Data with iCloud is built on top of the iCloud Drive service. The service pieces are not effected in any way and will continue to work. If and when the deprecated APIs are disabled in some future OS version, applications running on iOS 9 or 10 will continue to work.

However, App Store Review Guidelines say that apps cannot use deprecated technologies.

Update (2016-06-27): Daniel Pasco:

Last time I dove in to iCloud I lost a year of schedule and an evangelist eventually called me a masochist for continuing to try.

Apple File System (APFS)

Apple File System Guide (Hacker News):

Apple File System is a new, modern file system for iOS, OS X, tvOS and watchOS. It is optimized for Flash/SSD storage and features strong encryption, copy-on-write metadata, space sharing, cloning for files and directories, snapshots, fast directory sizing, atomic safe-save primitives, and improved file system fundamentals.

[…]

A Developer Preview of Apple File System is included in OS X 10.12. The APFS on-disk volume format is pre-release and subject to change. Apple plans to document and publish the APFS volume format when Apple File System is released in 2017.

Lee Hutchinson (Slashdot):

APFS also adds a copy-on-write metadata scheme that Apple calls “Crash Protection,” which aims to ensure that file system commits and writes to the file system journal stay in sync even if something happens during the write—like if the system loses power.

[…]

Snapshots and clones both are going to be available in APFS. Snapshots let you throw off a read-only instance of a file system at any given point in time; as the file system’s state diverges away from the snapshot, the changed blocks are saved as part of the snapshot. This is similar in concept to Microsoft’s shadow copies, and it’s an incredibly handy feature. There are obviously huge implications here in how Time Machine works—a true file system set of snapshots could totally replace the kludgy and aging mechanism of hard links that Time Machine builds and maintains.

Clones differ from snapshots in that clones are writable instead of read-only. According to the documentation, APFS can create file or directory clones—and like a proper next-generation file system, it does so instantly, rather than having to wait for data to be copied. A cloned file or directory stores the changes made between it and the original objects, giving you a writable, editable, point-in-time copy of a file or a directory. As the documentation points out, this is an easy way to create document revisions or do versioning of anything you might want to track.

Also interesting is the concept of “space sharing,” where multiple volumes can be created out of the same chunk of underlying physical space. This sounds on first glance a lot like enterprise-style thin provisioning, where you can do things like create four 1TB volumes on a single 1TB disk, and each volume grows as space is added to it. You can add physical storage to keep up with the volume’s growth without having to resize the logical volume.

Russ Bishop:

Although APFS does checksum metadata blocks it does not do anything to provide resilience for data blocks. That is a huge omission in a modern filesystem, a point I tried to politely but forcefully make in the File System Lab directly to a responsible engineer. I got the feeling that the APFS team is divided on the necessity of this feature and some people on the team would appreciate some ammo to help win the argument internally. I would encourage anyone who agrees to file radars ASAP requesting this feature.

[…]

I did get confirmation that Apple intends to provide an open specification in 2017 when the design is finalized. They also emphasized that the data structures are designed to be extended in backward and forward compatible ways so things like parity checks should be easily implemented even if they don’t make the first release.

Mark Dalrymple:

I love the sense of humor the low-level engineers have, as demonstrated by the -IHaveBeenWarnedThatAPFSIsPreReleaseAndThatIMayLoseData option.

Aymeric Barthe (via Benjamin Encz):

When it was first released, HFS+ did not support hard links, journaling, extended attributes, hot files, and online defragmentation. These features were gradually added with subsequent releases of Mac OS X. But they are basically hacked to death, which leads to a complicated, slow and not so reliable implementation.

[…]

Finally there is Bit Rot. Over time data stored on spinning hard disks or SSDs degrade and become incorrect. […] HFS+ lost a total of 28 files over the course of 6 years.

Adam Leventhal (via Hacker News):

By the next WWDC it seemed that Sun had been forgiven. ZFS was featured in the keynotes, it was on the developer disc handed out to attendees, and it was even mentioned on the Mac OS X Server website. Apple had been working on their port since 2006 and now it was functional enough to be put on full display. I took it for a spin myself; it was really real. The feature that everyone wanted (but most couldn’t say why) was coming!

[…]

I’m told by folks in Apple at the time that certain leads and managers preferred to build their own rather adopting external technology—even technology that was best of breed. They pitched their own project, an Apple project, that would bring modern filesystem technologies to Mac OS X. The design center for ZFS was servers, not laptops—and certainly not phones, tablets, and watches—his argument was likely that it would be better to start from scratch than adapt ZFS. Combined with the uncertainty above and, I’m told, no shortage of political savvy their arguments carried the day. Licensing FUD was thrown into the mix; even today folks at Apple see the ZFS license as nefarious and toxic in some way whereas the DTrace license works just fine for them. Note that both use the same license with the same grants and same restrictions. Maybe the technical arguments really were overwhelming (note however that ZFS was working internally on the iPhone), and maybe the risks really were insurmountable. I obviously have my own opinions, and think this was a great missed opportunity for the industry, but I never had the burden of weighing the totality of the facts and deciding. Nevertheless Apple put an end to its ZFS work; Apple’s from-scratch filesystem efforts were underway.

[…]

In the 7 years since ZFS development halted at Apple, they’ve worked on a variety of improvements in HFS and Core Storage, and hacked at at least two replacements for HFS that didn’t make it out the door. This week Apple announced their new filesystem, APFS, after 2 years in development. It’s not done; some features are still in development, and they’ve announced the ambitious goal of rolling it out to laptop, phone, watch, and tv within the next 18 months. At Sun we started ZFS in 2001. It shipped in 2005 and that was really the starting line, not the finish line.

Previously: ZFS, The Loss of ZFS.

See also: WWDC Session 701: Introducing Apple File System, ZFS data integrity explained, Trust, But Verify, Belt and Suspenders, Computational Skeuomorphism.

Update (2016-06-20): Adam Leventhal has written a series of posts based on his talks with Apple engineers Dominic Giampaolo and Eric Tamura at WWDC. Overview (Hacker News):

APFS, the Apple File System, was itself started in 2014 with Dominic as its lead engineer. It’s a stand-alone, from-scratch implementation (an earlier version of this post noted a dependency on Core Storage, but Dominic set me straight). I asked him about looking for inspiration in other modern file systems […] he was aware of them, but didn’t delve too deeply for fear, he said, of tainting himself.

Encryption, Snapshots, and Backup:

ZFS includes snapshots and serialization mechanisms that make it efficient to backup file systems or transfer file systems to a remote location. Will APFS work like that? Probably not[…]

[…]

Space sharing is more like an operational detail than a game changing feature. You can think of it like special folders with snapshot and encryption controls…

Space Efficiency and Clones:

Clones open the door for potential confusion. While copying a file may take up no space, so too deleting a file may free no space. Imagine trying to free space on your system, and needing to hunt down the last clone of a large file to actually get your space back.

Aaron Meurer:

Cloning fixes an important problem with hard links. The issue with hard links is that some applications will write “into” hard links, effectively changing both “copies” of the file, and other applications will “break” the link (copy on write).

[…]

It’s also worth pointing out that hard links typically share the same metadata (such as file permissions). APFS clones assumedly would be less strict about this (e.g., allow two clones of the same data have different file permissions).

Performance:

Apple controls the full stack including the SSD, FTL, and file system; they could have built something differentiated, optimizing this components to work together. What APFS does, however, is simply write in patterns known to be more easily handled by NAND.

[…]

APFS also focuses on latency; Apple’s number one goal is to avoid the beachball of doom. APFS addresses this with I/O QoS (quality of service) to prioritize accesses that are immediately visible to the user over background activity that doesn’t have the same time-constraints.

Data Integrity:

APFS removes the most common way of a user achieving local data redundancy: copying files. A copied file in APFS actually creates a lightweight clone with no duplicated data. Corruption of the underlying device would mean that both “copies” were damaged whereas with full copies localized data corruption would affect just one.

[…]

APFS checksums its own metadata but not user data. The justification for checksumming metadata is strong: there’s relatively not much of it (so the checksums don’t consume much storage) and losing metadata can cast a potentially huge shadow of data loss. If, for example, metadata for a top level directory is corrupted then potentially all data on the disk could be rendered inaccessible. ZFS duplicates metadata (and triple duplicates top-level metadata) for exactly this reason.

Explicitly not checksumming user data is a little more interesting. […] Apple engineers I spoke with claimed that bit rot was not a problem for users of their devices, but if your software can’t detect errors then you have no idea how your devices really perform in the field. ZFS has found data corruption on multi-million dollar storage arrays; I would be surprised if it didn’t find errors coming from TLC (i.e. the cheapest) NAND chips in some of Apple’s devices.

This is disappointing. EagleFiler has detected lots of bit rot for me over the years, including on SSDs, and I’ve seen lots of unviewable old JPEG files.

Conclusions:

Those are great goals that will benefit all Apple users, and based on the WWDC demos APFS seems to be on track (though the macOS Sierra beta isn’t quite as far along).

Update (2016-06-27): See also: TidBITS, comments on ArsTechnica, comments on Hacker News, Accidental Tech Podcast.

Update (2016-07-03): See also: Accidental Tech Podcast.

Git 2.9’s diff.compactionHeuristic

Git (via Hacker News):

If you’ve used Git, you’ve probably looked at a lot of diffs. Most of the time the diff shows exactly what you did: changed some lines, added some new ones, or took some away. But because Git generates the diff only from seeing the “before” and “after” states of your files, sometimes the way it shows the changes can be pretty confusing.

[…]

The author probably didn’t write the loop as two out-of-order halves. But because the outer lines of the old and new loops are the same, it’s equally valid to attribute the lines either way (it’s tempting to say that the problem can be solved by always starting the added lines at the top, but that only fixes this case; you’d have the opposite problem if the new loop were added after the old).

In 2.9, Git’s diff engine learned a new heuristic: it tries to keep hunk boundaries at blank lines, shifting the hunk “up” whenever the bottom of the hunk matches the bottom of the preceding context, until we hit a blank line.

It’s experimental, so you have to enable diff.compactionHeuristic with git config. Xcode 7.3.1 comes with Git 2.7.4, and the Xcode 8 beta comes with 2.8.1.

Cornerstone 3 Is Sandboxed

Zennaware:

The removal of color from the user interface played a significant role in meeting these goals. Color is now exclusively reserved for the indication of status, both for user interface elements, where blue is used consistently to include that something is on/selected, and for the version control status of user content (e.g. file status, text differences).

[…]

Cornerstone 3 implements sandboxing in order that we can continue to offer Cornerstone to Mac App Store customers. […] The implementation of sandboxing in Cornerstone was extremely challenging. While many other developers have either given up or taken the opportunity to remove their apps from the Mac App Store citing the difficulty of implementing sandboxing, we have invested heavily into re-engineering significant parts of Cornerstone to not just work within the confines of sandboxing, but to embrace it.

[…]

External compare tools can no longer be run directly from Cornerstone. Instead, the compare tool plugin for your tool of choice must be installed to Cornerstone’s application script folder using the new Compare Tools window (see the Compare Tools… command in the Cornerstone menu).

Looks like a huge update. They also sell direct, which is the only way to get the upgrade discount.

New Features Coming to Apple Music

Kirk McElhearn:

There were rumors that Connect was going to be deleted, but apparently Apple is going to try to make it work. It’s is a good idea for Connect to be in the For You section, instead of having its own tab.

[…]

Switch to Browse to see what’s new, and what’s not.

Again, there are text tabs in the window, rather than using a menu. New Music shows what’s new on Apple Music, Curated Playlists is what it says, then you see Top Charts, if you want to know what’s popular, and Genres takes you to a rather boring page of text links from which you can access a number of genres. The options within each genre haven’t changed, but the fonts have.

[…]

Here are a few screenshot of Apple Music on iOS; as you can see, they essentially reproduce the same elements as iTunes, just in a vertical layout. One problem I see is the size of the artwork for playlists and albums; it should be smaller, so you can see a bit more content.

Thursday, June 16, 2016

Early App Store Search Ads Fail

David Barnard and Paul Haddad are finding that when someone searches for their apps—by exact name—the App Store shows a search ad for a competing app at the top of the search results. This is exactly the sort of thing developers feared when search ads were announced.

Launch Center Pro
Calcbot

DivyB:

Not good - this is exactly what I thought Schiller said wouldn’t happen :(

Update (2016-06-16): @manolosavi is getting both an ad and an organic search result for the same app (via Preshit Deorukhkar). This is also bad because developers shouldn’t have to pay when they are already the top result.

John Siracusa:

Schiller talked about this in The Talk Show live. Short version from memory: this is allowed and benefits small devs more than big.

I’m very skeptical that this is a benefit for developers (who would have to outbid others just to protect their own app’s name as a search term) or for users (who expect what they search for to be at the top). Note, also, how the ad takes up more of the screen than the actual matching search result.

Nick Heer:

A very large company like Facebook could conceivably bid on all major social networking keywords, for example, and push down any competitors on the search results page. Or, in a more nefarious scenario, a well-funded low-quality or knockoff app could bid its way to the top of search. I’d hope Apple puts a stop to this; it would be inadequate to simply add “Report” buttons for ads that engage in this sort of behaviour.

Update (2016-06-17): Searching for PCalc shows an ad for PCalc Lite above the actual Calc.

Update (2016-06-20): Phil Schiller talked more search ads at The Talk Show Live.

Update (2016-06-21): Another example from Paul Haddad.

Update (2016-06-27): Rebecca Slatkin:

Watching ‘Introducing Search Ads’ so in a few years when all my indie developer friends go back to consulting I’ll know why.

“Because user experience is so important to us..”

I never thought Apple would ever give into making $ this way

Apple claims to possess a unique set of values and beliefs including raising the bar. Pretty sure this does the opposite.

Gus Mueller:

It’s like modern Apple though. Ask yourself, who does this help?

Daniel Jalkut:

I have asked, and can’t think of anybody but deep-pocketed developers with non-definitive search results.

Update (2016-07-02): Asher Vollmer (via Cabel Sasser):

An ad for 2048 shows up at the top of the @AppStore when you search for “Threes” in iOS 10.

This is exhausting.

Update (2016-07-10): Daniel Jalkut:

I am rarely as convinced that an Apple move is bad as I am with paid App Store placement. It defies my understanding of Apple’s values.

Update (2016-07-15): SplitMetrics:

The fact that the app in the first place got more clicks than the one on the 4th place was rather predictable. But the difference in those clicks is surprising: moving from the 4th place to the 1st increased the traffic share from 1% to 57%.

[…]

In both alternatives, the top app whether it was an ad or not achieved by far the most clicks on the ‘Get’ button. A ‘search ad’ app received 57% of clicks and the top app in a regular search result – 71%. The relatively small gap in CTR means that users don’t see the difference between sponsored and organic listings.

Update (2016-07-28): See also: David Chartier.

Update (2016-08-25): See also: Oisín Prendiville.

Update (2017-01-21): Dan Luu:

Google outbids Mozilla in the AppStore and shows up first for searches on “Firefox”.

CloudKit and Map Kit for Gatekeeper Apps

In 2011, I wrote:

Would anyone be surprised if future versions of Mac OS X made additional features and APIs available only to App Store apps?

In 2012, this came to pass:

It’s no longer possible to write a single app that takes advantage of the full range of Mac OS X features. Some APIs only work inside the Mac App Store. Others only work outside it. Presumably, this gap will widen as more new features are App Store–exclusive, while sandboxing places greater restrictions on what App Store apps are allowed to do.

In macOS 10.12, this will be less true. Developer ID apps outside the Mac App Store will be able to do almost everything:

Developer ID-signed apps can now take advantage of CloudKit, iCloud Keychain, iCloud Drive, remote (push) notifications, MapKit, and VPN entitlements.

I say “almost” because iCloud key-value storage and Core Data are not mentioned in the document, although my impression from the WWDC session is that they will be included. Of course, this assumes that Apple does not renege on this, as it did with Map Kit and El Capitan.

At WWDC, Apple is presenting the Mac App Store as the preferred way to distribute software. However, it still lags behind the iOS App Store in features. Developer ID apps have access to more APIs and more business models.

Andrew Pouliot:

Feels good to decouple the two, let MAS have its own value proposition apart from CloudKit, vice-versa.

Milen Dzhumerov:

Basically, they remove all the reasons for MAS (together with paying for direct Mac apps using Apple Pay).

Drew McCormack:

Question is whether they have just given up on it. Wouldn’t be the first time Apple just leaves something for dead.

The situation continues to get worse for unsigned apps. In macOS 10.12, the (graphical) preference to enable them has been removed, although there are still ways around that.

Gatekeeper Path Randomization

Jeff Johnson:

Under what circumstances does App Translocation occur? First, the app must have a com.apple.quarantine extended attribute. If you delete the quarantine xattr, then App Translocation does not occur, and the app will launch from where it was unarchived, like normal. Second, the app must be opened by Launch Services. This usually means Finder, but it can also mean open from Terminal, for example. If you launch the app executable directly from bash, on the other hand, App Translocation does not occur. Third, the app must not have been moved — by Finder. If you move the app, using Finder, from the app’s original unarchived location to another folder, even a subfolder, e.g., ~/Downloads/Test/, then App Translocation does not occur. However, if you move the app using mv from Terminal, then App Translocation will still occur. Normally you would move the app from ~/Downloads to /Applications, and that would cause the app to be launched from /Applications like normal, but the locations of the particular folders don’t seem to matter. The mere act of moving the app using Finder stops App Translocation from happening. Indeed, once you’ve moved the app once, it will no longer experience App Translocation again, even if you then move it back to ~/Downloads.

[…]

How does App Translocation affect us, as app developers? For the most part, the apps seem to work as before. After all, it shouldn’t matter whether your app is launched from /Applications, from ~/Downloads, or from the Xcode build folder. However, when the app runs from a read-only volume, it cannot update itself to a newer version. App Translocation prevents software update mechanisms such as Sparkle from working.

Jeff Johnson:

This is because the vulnerability that Apple is attempting to fix is what the presenter called a “repackaging” problem. The repackaging problem was illustrated at the end of Patrick Wardle’s paper, where he gives an example of a maliciously crafted disk image. It was clear from the session that Apple is concerned not just with Wardle’s dylib hijacking but also with any resources loaded external to the app bundle. Some apps will load external scripts or html, either of which could be employed in an attack. A locally loaded html page has access to the user’s file system, so it could wreak havoc if the page contains maliciously crafted JavaScript. App Translocation was designed to prevent malicious external resources from being loaded by changing the relative paths from the app bundle.

See also: WWDC Session 706: What’s New in Security.

Update (2016-06-17): Jeff Johnson:

The fundamental problem, the primary cause of both of the flaws I discovered, is that App Translocation happens automatically, without any awareness by or notification to the user, but App Translocation is turned off by a manual action by the user — moving the app bundle in Finder — again without any awareness or notification.

[…]

The more I learn about App Translocation, the more I feel that it is a bad idea with a botched implementation.

Update (2016-06-29): Jeff Johnson (tweet, Hacker News):

The problem with Gatekeeper Path Randomization is that copying applications to a read-only disk image will break functionality in many, if not most, existing applications. Perhaps most notable, features like automatic software updates (via Sparkle or similar mechanisms) will no longer work. Apple may not view this as an issue, given that GPR will be disabled once the user moves the application out of the Downloads folder. However, many users run applications from the Downloads folder, never moving them. This is especially common when a user is trying out an application prior to purchasing it, and an app that doesn’t work as expected due to GPR seems certain to lead to lost sales. Worse, even if the customer moves your app to their Applications folder, it may continue to be broken, depending on how your app is packaged.

[…]

Because the vast majority of apps do not load external resources relative to the app bundle path, only a very small percentage of apps are vulnerable to the attack that Gatekeeper Path Randomization is attempting to prevent. Nevertheless, Apple’s current implementation for GPR affects all apps outside the Mac App Store, and the negative impacts will be felt far and wide. While we believe there are several changes Apple can and should make, a simple workaround would be to allow software developers to opt out of Gatekeeper Path Randomization. If an app does not load external app-relative resources, GPR provides no benefit. The app should thus be able to put a key in its Info.plist specifying that GPR should not apply to the app.

Update (2016-06-30): Jeff Johnson:

I respect that Apple had a difficult choice. I’ve been thinking a lot about it, and I’m not sure exactly what mechanism I would use to protect against the repackaging problem. Gatekeeper Path Randomization is flawed, in my opinion, but there is no ideal solution. Any alternative I can think of would also have major downsides. This is why the Info.plist key is so important. The best way to prevent any problems or breakage caused by Gatekeeper Path Randomization is to avoid it completely. And we know it’s not necessary for most apps, because they aren’t vulnerable to repackaging, so there should be an easy way to indicate to Gatekeeper that an app doesn’t need it. It’s not naive to trust developers with an Info.plist key; the whole Gatekeeper and Developer ID system is based on trust. If that trust is violated, then Apple has the ability to revoke a developer’s signing cert, but thankfully the nuclear option has rarely been necessary.

Update (2016-07-11): The public beta version of DropDMG can now create signed disk images to avoid Gatekeeper path randomization.

Update (2016-07-21): DropDMG 3.4 is now available with support for creating signed .dmg disk images and .xip archives.

Update (2016-07-26): Jeff Johnson:

As I mentioned in an earlier blog post, there’s a new API for App Translocation in the 10.12 SDK: <Security/SecTranslocate.h>. However, if you’re sane, you’re not shipping your apps yet with the 10.12 SDK. So how can you detect App Translocation in your shipping apps? The trick is that since you can’t link against the new functions, you have to load them at runtime. I’ve provided sample code below.

Update (2016-08-18): Jeff Johnson:

Therefore, we’re faced with the odd situation where App Translocation affects our apps, but there’s no supported way to detect it, not even with the latest SDK. It’s unclear how this situation can be tenable. As we’ve seen, though, Apple works in mysterious ways. Thankfully, the SecTranslocate symbols still exist in the Security.framework binary, so the technique explained in my last blog post continues to work. The Dude abides.

Update (2016-10-07): Jeff Johnson:

In an earlier post I mentioned that the I stumbled upon a different place where a kind of API for App Translocation still exists in macOS 10.12 Sierra: the security command-line tool. It’s not in the man page, so don’t bother to RTFM, but you will find it at the end of the usage information, when you simply type security into Terminal[…]

Update (2017-02-18): Johannes Tiefenbrunner:

If you put these two things together it becomes impossible to create permanent (“Forever”) rules in Little Snitch for apps that are still in quarantine, because each time they are launched they will be launched from a different path.

Update (2018-06-21): Jeff Johnson:

The kicker is that this App Translocation issue only occurred on Sierra. On High Sierra, the SafariServices API calls work even when the app is running from the Downloads folder and undergoing App Translocation. So it appears that SafariServices on Sierra was failing to account for App Translocation, and that bug was fixed in High Sierra.

SiriKit

Andrew Cunningham (Hacker News):

Third parties can use Siri in six different kinds of apps, though those applications encompass a wide range of common and popular App Store offerings: audio and video calling apps; messaging apps; payment apps; apps that allow searching through photo libraries; workout apps; and ride booking apps.

That covers a lot of ground, but there’s a lot that’s missing: music and video apps like Spotify and Netflix, mapping apps like Google Maps, and third-party to-do list apps don’t fall under any of these umbrellas, possibly because they conflict too directly in some cases with Apple apps and services like Apple Maps and Apple Music (Apple Maps in particular is becoming more deeply integrated into the OS with every new update). Hopefully Siri will become more useful to a wider variety of apps later on, but for now the first-party apps are definitely still in a privileged position.

And Apple didn’t draw too much attention to it yesterday, but these Siri capabilities are only available in iOS 10, not in the newly Siri-fied macOS Sierra.

Clark Goble:

This is so inexplicable. Why can’t I pass a search to a weather app to see the weather for a city? Why can’t I start a podcast in a 3rd party podcast app like Downcast. (Honestly, that’s the time I’m using my phone in the car -- much more than sending SMS by voice) Why can’t I ask Daylite to dial a contact? Why can’t I get tomorrow weather for a city I’m traveling to from my weather app of choice?

Honestly the places they open are least useful to me.

Tim Schmitz:

Although I’m happy there’s finally a developer API for Siri, I’m disappointed that it’s limited to a fairly small set of applications for now. As a user, what I want most is for my favorite shopping and to-do list apps to get Siri integration, something that won’t be possible with what Apple announced today.

Husain Sumra:

When asked why Siri’s API is limited to certain kinds of apps, like ride-hailing services like Uber or messaging, Federighi and Schiller once again talked about Apple’s baseline philosophy. Federighi said the decision to go with those apps types was made because Siri largely understands the domains of messaging and requesting purchases, making it easier to give the keys to developers and ensure a great user experience. He also said that Apple is working to expand Siri’s familiarity with certain domains over time.

I think this is fundamentally the wrong approach. Imagine how useful AppleScript would be if you could only use Apple’s standard suites. Adoption and innovation would be greatly slowed, and lots of interesting things would never be possible. Amazon’s approach of letting Alexa Skills add their own terminology makes a lot more sense. I care much more about functionality and reliability than about having lots of variant, more natural syntaxes for the basic queries.

Secondly, I’m disappointed that there is still no easy way to simply save a bunch of dictated text as a reminder (to be imported into OmniFocus). Most of the time, this is all I want to do when hands-free. Instead, if I say “Remember to,” Siri will try to parse the following text and inevitably mess it up.

See also: WWDC Session 217: Introducing SiriKit.

Update (2016-06-16): Joe Rosensteel:

Limitations of SiriKit are baffling when you think about Siri’s universal search for media on the Apple TV.

Update (2016-07-17): Steven McMurray:

In this article we’ll dive deep into how Siri works, the fundamentals of implementing Siri, and how to design a great user experience with Apple’s own personal assistant.

Update (2016-07-18): Ruffin Bailey:

I understand that this allows us to skip translation, but is that really that big a deal? Smart folk are internationalizing (there’s a better word for that; sorry) their apps now, translating labels and other text throughout. I also get that Apple might do a better job with grammar, so that there are lots of ways to get across a command in natural speech rather than forcing a strange, app-specific grammar. I don’t feel that’s a big win.

Microsoft Buys LinkedIn

Michael J. de la Merced and Leslie Picker:

How generous is Microsoft’s takeover bid? It puts LinkedIn’s enterprise value at 79 times the social network’s earnings before interest, taxes, depreciation and amortization, or Ebitda, for the 12 months that ended on March 31.

On the basis of that multiple, the transaction is more expensive than any big internet deal paid with cash, according to data compiled by Bloomberg.

Microsoft is paying $220 for each of LinkedIn’s monthly active users. By comparison, when Facebook acquired WhatsApp for $19 billion two years ago, it spent $40 for every user.

Ben Thompson:

LinkedIn builds a bridge for their productivity business to a new world centered around end users, not corporations, and it may even give Microsoft’s inevitable compete products a head start. Imagine this: instead of simply moving Active Directory to the cloud, Microsoft is potentially making LinkedIn the central repository of identity for all business-based interactions: chat, email, and more, and it’s an identity that endures for an end user’s professional life, because it’s managed by the user, not by their transient employer. It’s genuinely exciting, and shocking though it may have been, I think that’s worth $26 billion.

John Gruber:

That’s a lot of money but it seems like an obvious fit. Also, Nadella was probably tired of getting all those LinkedIn emails.

Nick Heer:

Microsoft’s track record with multibillion-dollar acquisitions is, well, terrible. But why would Microsoft want to own the world’s second uncoolest social network — the first, of course, being Google+ — when it’s effectively a glorified and unprofitable résumé hosting service?

Wednesday, June 15, 2016

WWDC 2016 Links

Apple:

General Coverage:

Specific Topics:

Podcasts:

Previously: Pre-WWDC Thoughts, WWDC 2015 Links.

Sunday, June 12, 2016

Facebook Threatens to Delete Photos If Users Don’t Download Moments App

Jacob Kastrenakes (via Hamish Sanderson):

That syncing tool has now been moved out of the core Facebook app and into the photo app Moments. Facebook made it clear that this would happen — and in fact it happened months back, seemingly without much pushback. What Facebook was less clear about was what would happen to photos that had previously been synced.

At the time, Facebook mentioned that people would be given the chance to download their pictures in a zip file if they didn’t want to move over to Moments. Now it’s clear that offering is because Facebook intends to completely delete synced photos if people don’t download this new app.

Mark Wilson (via Slashdot):

This has understandably led to panic installations of Moments as people sought to protect the photos that have been automatically synchronized from their phone. It’s important to note that it is only these synced photos that are at risk, but it’s clear that there is an element of confusion about what Facebook is planning to delete. The email the company is sending out to users[…] The email is very poorly worded, and almost has the feeling of spam to it.

Twitter Account Hacked via SIM Reset

Kate Conger (via John Gruber, Slashdot):

Even though several huge data breaches have recently exposed hundreds of millions of social media login credentials online, users aren’t re-setting their passwords — which means you’ll probably continue to see celebrities’ social media accounts getting hijacked.

[…]

After regaining control of his Twitter account, Mckesson explained that the hacker or hackers were able to take over by convincing Verizon to reset his SIM. With the SIM reset, the person responsible was able to receive text messages intended for Mckesson and therefore bypass the two-factor authentication the activist used to keep his account secure.

[…]

Passwords for tens of millions of Twitter accounts appeared online for sale this week, following the hacks of accounts belonging to Katy Perry, Ev Williams, Mark Zuckerberg, Drake and others. Although it’s possible that some of their passwords were included in the auctioned database, it’s more likely that their accounts were compromised because they reused passwords from other breached websites like LinkedIn, Myspace and Tumblr.

Nicole Nguyen:

With the last four digits of Mckesson’s Social Security number, they were able to gain full access to his Verizon account and changed the SIM, which redirected texts to a different device.

The hackers didn’t even need his account’s passwords. They could simply reset passwords to trigger two-factor authentication.

[…]

In a blog post, the FTC’s Chief Technologist Lorrie Cranor described how you can add an extra layer of security with your mobile carrier.

Without the proper carrier settings, two-factor is actually less secure than only using a password. AT&T customers should enable the “Extra security” option so that accessing the account always requires a 4-digit PIN other than the SSN.

Previously: Seven Hundred Million.

Update (2016-06-12): Glenn Fleishman:

Companies retain SMS as an option because of the customer-support burden: it’s easier to get someone to type in a code sent as a text message than to download, install, configure, and use an authenticator app. But you would think the time is ripe for companies to allow expert users to disable SMS as a backup option, especially since many sites pair turning on 2FA with creating a set of backup, one-time use passwords intended to restore access if one loses access to the authentication app that can generate the appropriate code.

Thursday, June 9, 2016

Pre-WWDC App Store Changes

Apple:

Starting this summer, you’ll be able to participate in the Search Ads beta and see the ads in action.

[…]

We’re opening auto-renewable subscriptions to all app categories including games, increasing developer revenue for eligible subscriptions after one year, providing greater pricing flexibility, and more.

Apple:

If you have an app with auto-renewable subscriptions, you have the option of offering free trials to new users. The length of the subscription determines how long the free trial can be. For example, for a monthly subscription you can offer users a 7-day or 1-month free trial. When users sign up for a subscription with a free trial, their subscription will begin immediately but they won’t be billed until the free trial period is over. They will then continue to be billed on a recurring basis, unless they turn off auto-renewal.

Apple:

Within each subscription group, you can offer different levels of service (for example, basic, premium, and pro memberships) as well as durations (such as weekly, monthly, or annually).

Starting this fall, users will be able to move easily between service levels, and choose to upgrade, downgrade, or crossgrade (that is, move to another identically ranked auto-renewable subscription in the same subscription group) as often as they like.

Apple:

After a subscriber completes one year of paid service of your auto-renewable subscription, you automatically receive 85% of the subscription price, minus applicable taxes, on the subscriber’s subsequent renewals.

Apple:

Rather than needing to code your own subscription management UI, your app can open the following URL:

https://buy.itunes.apple.com/WebObjects/MZFinance.woa/wa/manageSubscriptions

Opening this URL launches iTunes or iTunes Store, and then displays the Manage Subscription page.

John Gruber:

Until now, subscription pricing was reserved for apps that served media content: streaming audio and video, news, etc. Apple is now opening it to apps from any category, which effectively solves the problems of recurring revenue and free trials.

[…]

This could be the change that makes the market for professional-caliber iPad apps possible. On the Mac, there has long been a tradition of paying a large amount of money for a pro app, then paying a smaller amount of money for major updates. The App Store has never allowed for that sort of upgrade pricing — but upgrade pricing is what enabled ongoing continuous development of pro software. Paying for each major new version, however, is arguably a relic of the age when software came in physical boxes. Subscription-based pricing — “software as a service (SaaS)” — is the modern equivalent. That’s the route both Microsoft and Adobe have taken. In the old world of boxed software and installation disks, the natural interval was the version. In today’s world where everything is a download, months or years are more natural payment intervals.

[…]

I think all serious productivity apps in the App Store should and will switch to subscription pricing.

There is uncertainty, however, because Apple’s statements seem deliberately worded to exclude some apps from subscription pricing:

In a sidebar titled “Types of Auto-Renewable Subscriptions”, Apple lists only two, “Content” and “Services” […] Professional apps that require “a lot of maintenance of new features and versions” don’t fit either of those categories. Would Twitter clients like Tweetbot and Twitterrific qualify for subscription pricing? After talking to Schiller yesterday, I thought so. Now, I don’t know. Developers are definitely confused.

It’s also not clear whether Apple will require, as it does with in-app purchases, that the app be useful without a subscription.

It’s hard to predict the effects of subscriptions without knowing the answers to these key questions. I’m cautiously optimistic because Apple is finally trying to address the problems of trials and sustainability. On the other hand, you could look at it as: this is Apple’s answer. There definitely won’t be true free trials or upgrade pricing any time soon—or, probably, ever.

The 85/15 revenue split after a year is also encouraging, not only because Apple is acknowledging that “the developer is doing most of the work,” but also because it opens the door to other fee adjustments in the future. I think there are a lot of cases where something lower than 30% would make sense. Interestingly, Google has already one-upped Apple and gone to 85/15 for the first year, too.

Assuming that any app could use subscriptions without restrictions, what would that mean? Subscriptions are definitely controversial. Customers generally don’t like them, though they may end up being the least bad option. One issue is the loss of control. It feels better to look at what’s in a new version and decide whether it’s worth it, rather than keep paying just to keep using what you already have. However, the trend is clearly toward everyone running the latest version, rather than supporting old versions. Simply maintaining an app takes work, and it makes sense for payments to reflect that. The other main issue is the overall cost. Subscriptions can really add up, and people will need to think more carefully about budgeting and saving in order to maintain access to key apps if their circumstances change.

From the developer side, recurring revenue is key, but will customers be reluctant to sign up? Even as someone who likes to pay for apps, I think twice about subscriptions. In a competitive marketplace, will customers find a subscription product attractive next to a competing one that is a one-time purchase? Will it help that only the subscription product can have a trial? How often will customers be able to try the app? How can a developer figure out a fair ongoing price up front, when it’s not known how the app’s launch will go or what path its future development will take? Will providing your own server and implementing subscriptions prove to be a difficult hurdle? I’m surprised that Apple does not provide a way to handle this automatically.

How will apps transition to subscriptions? Will they wait until when the next major upgrade would have been? How will existing customers react? Will they get special consideration? What will happen to the corresponding direct sale versions of the apps? Will they continue to be available with traditional upgrade pricing? Or will they try to honor the App Store subscription? Or offer a parallel subscription using a different payment processor?

I think the best thing that can be said for subscriptions is that they’re honest and mostly align everyone’s incentives properly. Customers will essentially vote with their wallets, on an ongoing basis. Developers who maintain and improve their apps will get recurring revenue. Apple will get more revenue when it steers customers to good apps. Over time, more of the money will flow to the apps that people actually like and use. My guess is that the average customer will end up spending more money on fewer paid apps. Some apps will become more sustainable, but others will be culled.

It remains to be seen how the search ads will work in practice, but my overall reaction is negative. An iPhone screen currently only displays two App Store search results at once. Now, the top one of those will be an ad, although customers may not realize it’s an ad. In theory, search ads could help a quality unknown app gain traction. But indie developers fear that big companies will use big ad budgets to tilt the playing field. And everyone may feel compelled to pay protection money just to keep their app in the search results for its own name. Lastly, everyone wanted Apple to fix search. Not only do search ads not do this, but they seem to give Apple a perverse incentive not to fix it.

These are clearly the biggest changes since the dawn of the App Store. I see it as a good sign that something is happening. However, much remains to be done. The store apps themselves need work, as do the underlying OS services. The Mac App Store still lags behind, without support for gifting apps or TestFlight, and the sandbox remains a mess.

More personally, although I have enjoyed quicker review times than in the past, I continue to encounter problems simply updating my apps. My last two updates were held up for two weeks by a backend store bug that prevented the packages from being uploaded with Application Loader.

Then, when one of them was finally ready for sale, I had to pull it from the store. Apple had incorrectly codesigned it so that customers could not even launch it. Why doesn’t Apple have automated testing in place to make sure that the app file that downloads is actually valid? This is a terrible first impression for new customers. And for existing customers, one day the app was working for them, and the next it was not.

After several days and failed attempts, it appears that the right people at Apple are now aware of the problem with my app. Meanwhile, I am fortunate to be able to issue customers licenses to the direct sale version so that they can actually use the app that they paid for. I’m confident that my app download will eventually be fixed, but the important point is that problems like these remain common so many years after the store’s launch. Instead of releases becoming routine, there always seems to be a new frustration.

See also:

Previously:

Update (2016-06-10): Nick Heer:

It definitely won’t work for every app, though, or even every niche. Once the uncertainty about which apps are eligible for subscriptions is cleared up, I think there are going to be specific qualities that justify a maintenance-style subscription: quality apps for a particular audience, made by indie developers who issue regular updates. And I think it needs all of those factors. Users aren’t going to pay for an app maintenance subscription if the app is of mediocre quality, or if the developer is a huge company, or if it is infrequently updated — or, at least, does not have some sort of ongoing justification for its subscription cost.

Glenn Fleishman:

This article has been updated to reflect clarification from Apple that apps approved to use subscriptions can require a subscription to use the app at all.

John Gruber:

In short, we don’t have all the answers we need yet. But Apple is aware of the questions.

Kyle Baxter:

Glad Apple’s aware of uncertainty re subscriptions and types of apps. But shouldn’t they’ve had an answer prior?

It’s kind of… weird that they wouldn’t have thought through that prior to making the announcement.

Riccardo Mori:

Among the apps I’ve bought over the years, a lot of them are quality apps I love and enjoy using, but I don’t necessarily use them all the time. […] I haven’t minded paying the occasional extra for a paid update or for the in-app purchase that unlocked more photo filters or editing features. But in the extreme case that all app developers behind these apps moved to a subscription-based pricing, without offering alternatives, I would be forced into a position I really don’t like: having to decide which app stays on my devices and which one has to go.

Manton Reece:

I hate that Apple has the power to reject our business model for a potential app. I’m now leaning more to the idea that Apple should approve nearly everything and let customers decide on the value. But there is a difference between maintenance of an app vs. a web service, and the services that are clearly appropriate for subscriptions will be the most successful apps using this new model.

Update (2016-06-11): See also:

Update (2016-06-12): Luc Vandal:

I wonder how many top class apps will switch to a subscription model. Considering it for @screensvnc but many questions still unanswered.

It’d be much better with paid upgrades IMHO in that regard but Apple thinks subscriptions are the way to go…

Vlad Savov:

The way most of us treat app purchases today is akin to buying a pair of shoes and expecting the manufacturer to provide free polishing and repairs. Forever.

The truth is that the economics of the App Store have never really made sense. We are paying negligible amounts once while expecting continuous development and improvement way off into the future.

Ole Zorn:

I still kinda want to decide for myself how long I’m wearing an old pair of shoes, and when I’m getting a new one.

Small developers shouldn’t make the mistake of thinking their app is like electricity, when it’s actually more like a pair of shoes.

See also: The Talk Show.

Update (2016-06-16): Matthew Bischoff:

Really odd that the Subscription session [at WWDC] didn’t cover any of the ongoing conversations about what’s acceptable as a subscription.

Scott Edwards:

During the live show with @gruber Schiller said they want everyone to use it, but are just leaving themselves wiggle room.

Marco Tabini:

Hard to account for “wiggle room” when you’re deciding to invest in building a new app, though.

Update (2016-06-20): Phil Schiller talked more about subscriptions and search ads at The Talk Show Live.

Eliminating iTunes Store Music Downloads

Mitchel Broussard:

New sources have come forth claiming that Apple is in fact aiming to phase out digital music downloads on iTunes, despite the fact that Apple rep Tom Neumayr specifically stated such rumors were “not true” in May.

[…]

According to the sources, Apple might be gearing up for an iTunes revamp that would introduce software architecture with the ability for the company to “more easily drop iTunes music downloads” down the road. This would allow Apple to subtly shift the service towards the streaming and radio side of things in the event that paid music downloads drop off precipitously.

This would be very disappointing, if true.

Update (2016-06-16): Eddie Cue:

There’s no end date, and as a matter of fact, they should all be surprised and thankful to the results that they’re seeing because our music iTunes business is doing very well. Downloads weren’t growing, and certainly are not going to grow again, but it’s not declining anywhere near as fast as any of them predicted or thought it would. There are a lot of people who download music and are happy with it and they’re not moving towards subscriptions.

Update (2017-12-11): Chance Miller:

Digital Music News today reports that Apple is “on schedule” to rid the iTunes Store Music Store of downloads by 2019.

The report explains that the move to phase out music downloads has been in place since 2016, but in a statement to 9to5Mac, Apple denies the report and simply says “it’s not true.”

Update (2018-01-29): Binyamin Goldman:

Apple’s plan to replace digital music sales with Apple Music will be completed by late 2019, according to our sources. Users who own digital music will have their libraries merged with Apple Music. Users not on the service will be encouraged to join, however, if they choose not to they will still be able to listen to their library.

Wednesday, June 8, 2016

New Sketch 4.0 Licensing Model

Bohemian Coding (tweet, Hacker News):

Until now, it has been our custom to release a major version of Sketch (2.0, 3.0, etc) every couple of years as a paid upgrade, with minor updates (3.1, 3.2, etc) for free in between. While this is a very common model for software, we don’t think it is fair: customers who have purchased an app closer to the original release date get free updates for longer than someone who bought the same product halfway through its release cycle. We think a fairer approach is for everybody to get the same period of free updates, no matter when they purchased the app.

We would also prefer to ship big updates and improvements frequently, when they’re ready, rather than waiting for the next major version to come around in order to entice people to upgrade. As a consequence, we intend to remove the distinction between major and minor releases altogether.

They are essentially doubling the price from $99 every two years to every one year, but it’s not really a subscription because if you stop paying you can keep using the version that you have.

Pre-WWDC Thoughts

Chris Adamson:

The only good desktop Mac available today is the top-of-the-line iMac. Everything else is a goddamned embarrassment: terrible values for the price and clearly products of neglect. In fact, I noticed yesterday that Ramjet is now offering tower-style Mac Pro 5,1’s retrofitted with new CPUs and 1 TB SSDs, and they are arguably more compelling than Apple’s brand-new Mac Pros (and cost $2,000 less!). It takes you back to the days when Power Computing’s Mac clones made a mockery of 1995 Apple’s Performas and 5000-whatever models.

[…]

Shazam and Twitterrific are best-of-breed apps on the iPhone, but their Watch apps are straight-up non-functional. Do we think the developers suddenly lost their mojo, or is it that the Watch APIs and internal systems simply do not work reliably enough to build third-party apps atop?

Andrew Cunningham:

Of those, the MacBook and the iMacs are both fine; they were updated fairly recently and there’s not a whole lot you could change about either machine with current technology. But the “Pro” machines especially are notable in a bad way. Mac designs have always lasted for longer than iPhone and iPad designs and Apple’s release schedule is at least partially determined by Intel’s, AMD’s, and Nvidia’s, but that’s a long time to go without revisiting designs.

Rumors suggest Apple will be paying some much-needed attention to its Mac lineup later this year, possibly at WWDC and possibly in the months after. In the meantime, let’s walk through features Apple could add to the platform and things it could change about individual Macs to keep pace with and get back on top of the industry this company once led.

[…]

A totally redesigned Mac Pro was announced with much fanfare at WWDC in 2013, three years after the old “cheese grater” Mac Pro got its last significant update.

Jason Snell:

Reverting back to calling the Mac operating system “Mac OS” is high on my wish list. I’d prefer that particular style to some newfangled “macOS”, but I’ll take that if it’s the only way to get the Mac back in the name of the operating system that powers it. I’d like the “X” consigned to the ash heap of history. I’m okay with the idea of place names continuing as the monikers for Mac OS releases, but I’d prefer Apple just go to sequential numbering as it has with its other platforms—starting with Mac OS 11 this fall.

[…]

More importantly, it’s way past time for both the Mac Pro and the MacBook Pro to receive updates. The Mac Pro hasn’t been updated in ages, to the point where it’s actually embarrassing that Apple’s still charging full price for such old technology.

Steven Frank:

WWDC pipe dreams: fix Finder, fix iTunes, fix App Store, stop iOSification of Mac OS, pro hardware with top-end specs and ports intact

John Siracusa:

…and a new file system.

Mark Sullivan:

Perhaps most importantly, Apple is likely to finally release a software development kit (SDK) to developers so that they can integrate Siri into their own apps.

Rohan Naravane:

In an attempt to keep the user experience in check, Apple has always given its own apps the first priority. For example, a hyperlinked email address in a messaging app when clicked, will open the Mail app, and not Gmail. Or when a location is shared on WhatsApp, it is Apple Maps that opens up by default, not Google Maps. Although past experience doesn’t suggest Apple will let people choose their default app preferences, letting Siri roam beyond the walled garden can provide 3rd party apps a level playing field at least with voice control.

Here are five things I’ll be glad to say to Siri once developers are given access, effectively bypassing some of Apple’s mediocre service offerings.

Nick Heer:

Of all the things I wish to see at WWDC next week, improvements to search — across the board — rank highly on my proverbial list. The company has come a long way in a very short period of time, but search still isn’t good enough; searching on both the App Store and Maps, in particular, remain among the most frustrating experiences on any of Apple’s platforms.

Dan Moren:

We spend a lot of time navigating through the lock and home screens on our iOS devices, and rather than them just being way stations for us to pass through en route to our apps, it’d be great if they offered a little more utility along the way.

[…]

Cut, copy, paste and text selection was missing for a long time on the iPhone. Those text-handling features didn’t arrive until iOS 3, but they’ve remained largely unchanged since then. The problem is that they’ve become increasingly finicky. Trying to select just the portion of a web page I want to copy has turned into a battle for me, trying to seize one of those blue text-selector handles and not having it snap away because the software thinks it knows what I want.

Jason Snell:

iOS needs to handle files better, specifically on external drives. I should be able to attach an SD card (or even a USB hard drive) full of files to my iPad Pro and import the files I need; right now, if it’s not a video or audio file, I’m out of luck.

Gus Mueller:

A Mac OS upgrade which is about bug fixes and minor features. Maybe even moving to an 18 month upgrade cycle instead of a yearly one.

Erica Sadun:

My answer is the same as it’s been for years: “Bug fixes and security enhancements.” I’m a bit over the yearly update cycle.

David:

I’d just like an Xcode, tool-chain that actually works.

As for me, the number one thing I want from WWDC is for Apple to focus on fixing bugs (in the OSes, apps, services, and Xcode).

Update (2016-06-09): Tim Schmitz:

WWDC kicks off in just a few days, which means it's prediction time. This year I've got one that's a little off the wall: Not only will there be an API for Siri, but it'll be iCloud-based and take advantage of Swift on the server.

Swift Configuration and “then”

Soroush Khanlou:

I toyed around with a few ways to fix this issue, including making the instance variables optional and filling in the defaults at the usage site, but that ended up being just as clunky as the big initializer. What I settled on was: instead of making the variables optional, I made them mutable. That solves quite a few problems.

[…]

The then extension is a really useful library to have in your app. […] Since everything in Cocoa Touch inherits from NSObject, you can now use this function to configure lots of types, and you can do so at the declaration of the property. Swift will let you initialize things in-line (as long as they’re effectively a one line expression) […]

Swift Mistakes

Zev Eisenberg:

Even though I’ve been keeping up with Swift blog posts, my first Swift app was a port of an old and crusty Objective-C code base, and I carried over a lot of old habits. In looking back over it, I found many small- to medium-sized changes that make the code cleaner and more Swifty. Maybe I can help you avoid similar mistakes.

[…]

Sadly, you can’t make protocol conformances private, but otherwise you should be using private extensions wherever you aren’t required to make them public.

[…]

There are certainly cases where direct comparison to nil are appropriate, but Swift’s language features make it pretty rare in my experience.

[…]

Since you can extend any type, it’s convenient to make extension files for your utilities on CGRect, CGPoint, UIInterfaceOrientation, and other types that couldn’t be extended when they were lowly C structs. So don’t litter your main code files with one-off domain-specific utilities on system types. Put them in a separate file and write a couple of unit tests just for that file.

Daniel Jalkut:

At first, I didn’t think too much of this bridging header. Sure, I want to access my Objective C files from Swift. Which ones? Why not all of them?

[…]

My takeaway is to accept that it is fundamentally unsafe to interface with Objective C classes whose headers have not been audited for nullability.

Tuesday, June 7, 2016

Cartography Comparison: Google Maps & Apple Maps

Justin O’Beirne:

At its heart, this series of essays is a comparison of the current state of Google’s and Apple’s cartography. But it’s also something more: an exploration into all of the tradeoffs that go into designing and making maps such as these.

[…]

Apple is labeling more cities. And Apple is also showing more roads. But Google has more road labels.

[…]

That means that out of a combined total of 35 unique places on both maps, the two maps have only 5 places in common.

[…]

It’s clear that Google thinks transit is important, while Apple thinks that airports, hospitals, and landmarks are important.

He does not look at rural areas, on the grounds that “the things that are labeled are the area’s only things.” Nonetheless, I’ve still noticed major differences, with Apple Maps typically omitting smaller roads, in favor of showing nothing, unless you really zoom in. In urban areas, I like that Google always shows the transit stations.

Nick Heer:

As noted earlier, I find the most striking difference to be search. Google Maps is very good at it; Apple Maps isn’t. The latter is getting better, insomuch as it less-frequently shows me results in Delaware and Washington D.C. instead of Calgary, but Google is still leaps-and-bounds ahead.

Khoi Vinh:

O’Beirne refrains from calling one or the other superior, but for me, the clear difference is that Apple Maps is concerned with cartographic integrity where Google Maps is concerned with the experience of using the application. That is to say, at most given zoom levels, Apple Maps presents formally better maps, but holistically, Google Maps presents more of the right information at the right time. Which is consistent with what animates each company: Apple is focused on beauty and elegance, and Google is focused on information delivery.

I still greatly prefer Google Maps, both for its design and map data. However, Google has made some changes recently that seem like regressions.

Previously: Google Maps & Label Readability.

Update (2016-06-08): Lee Bennett:

Two huge reasons I prefer Google Maps: Street View and single handed (one finger) zooming.

E.W. Scripps Buys Podcast Company Stitcher

John Gruber:

Midroll owning Stitcher is not good for the podcast ecosystem. Stitcher is popular, but my show is not on Stitcher because Stitcher re-hosts the audio, compresses it to hell, and unless you opt out, inserts their own ads. That’s not how podcasting is supposed to work. I firmly believe podcasting should be open, like the web.

Marco Arment:

I’ve never been more proud to be operating a large podcast app that’s built on standard RSS, open access, and standard playback of podcasters’ original files directly from their servers, with no garbage ads being inserted, no behavioral tracking for advertisers, no proprietary lock-in, and absolutely no requirements that podcasters register with me, do anything differently, lose any control, agree to any terms, or even be aware of my app at all to be played, shared, and promoted in it.

Don’t let proprietary podcast platforms convince you that we need them.

Stephen Hackett:

I’m not as worried with services that re-host files — like Google Play — as Gruber is, but I agree that locking advertising to a select platform is bad for the industry.

Update (2016-06-07): Ben Thompson:

Midroll sells ads for over 200 podcasts, including some of the most popular ones like WTF with Marc Maron and the Bill Simmons Podcast. The not so-secret reality about podcast ads, though, are that advertisers are quite concentrated: a FiveThirtyEight intern heroically listened to the top 100 shows on the iTunes chart and counted 186 ads; 35 percent of them were from five companies. More tellingly, nearly all of the ads were of the direct marketing variety.

[…]

The real money in TV and especially radio is brand advertising; brand advertising is focused on building affinity for a purchase that will happen at some indefinite point in the future, so the focus is less on conversion and more on targeting: knowing in broad strokes who is listening to an ad, and exactly how many. For podcasting to ever be a true moneymaker it has to tap into that — and that means changing the fundamental nature of the product.

[…]

I know this breaks the modern concept of podcasting, and power users with tens of subscriptions in their podcasting player of choice will be annoyed if they have to download multiple apps. Often, though, a solution that works for power users is actually prohibitive for normal users, and the other solution — a Facebook of podcasts — would be worse for everyone.

Update (2016-06-08): Michael Rockwell:

The best we can hope is that Midroll treats Stitcher users like Neilson households — a small sample size that can be used to represent all podcast listeners.

Mutexes and Closure Capture in Swift

Matt Gallagher:

Using a mutex in Swift isn’t particularly difficult but I’m going to use the topic to highlight a subtle performance nuisance in Swift: dynamic heap allocation during closure capture. We want our mutex to be fast but passing a closure to execute inside a mutex can reduce the performance by a factor of 10 due to memory allocation overhead.

[…]

One of the advantages with closure capture is how effortless it seems. Items inside the closure have the same names inside and outside the closure and the connection between the two is obvious. When we avoid closure capture and instead try to pass all values in as parameters, we’re forced to either rename all our variables or shadow names – neither of which helps comprehension – and we still run the risk of accidentally capturing a variable and losing all efficiency anyway.

[…]

It turns out that despite being a private function in the same file – where Swift can fully inline the function – Swift is not eliminating redundant retains and releases on the the PThreadMutex parameter (which is a class type to avoid breaking the pthread_mutex_t by copying it).

We can force the compiler to avoid these retains and releases by making the function an extension on PThreadMutex, rather than a free function[…]

[…]

But despite the speed increase, using a semaphore for a mutex is probably not the best approach for a general mutex. Semaphores are prone to a form of priority inversion.

Long-term, Swift should be able to keep @noescape closures on the stack.

Monday, June 6, 2016

GCD’s Main Queue vs. Main Thread

Benjamin Encz (tweet):

Technically I think this is a MapKit / VektorKit bug, Apple’s UI frameworks typically guarantee to work correctly when being called from the main thread, no part of the documentation mentions that code needs to be executed on the main queue.

[…]

By combining GCD and Cocoa’s NSThread API we are drilling through the abstraction that GCD should provide in the first place. APIs that fully rely on GCD and ignore the presence of the underlying threads can run into problems if you call them on the main thread but not the main queue. This means, especially when calling into other frameworks, we should prefer using dispatch_queue_set_specific and dispatch_get_specific to check if our code is running on the main queue over using NSThread.isMainThread().

It’s hard to get this right because of reentrancy, though.

Avoiding the Overuse of @objc in Swift

Swift 2.2’s #selector syntax introduces a new compiler error:

Argument of '#selector' refers to a method that is not exposed to Objective-C.

Fix-it   Add '@objc' to expose this method to Objective-C

Jesse Squires:

We cannot simply add @objc to this method in the original ViewControllerType protocol for a number of reasons. If we do, then the entire protocol needs to be marked @objc, which means[…] Our example here is simple, but imagine a much more complex object graph that makes heavy use of Swift’s value types and a hierarchy of three protocols with this one in the middle. Introducing @objc as the fix-it suggests would break the entire world in our app.

[…]

We can decompose this protocol by separating out all of the @objc code into its own protocol. Then, we can use protocol composition to reunite them.

Converting Old Videotapes

David Pogue:

As it turns out, YesVideo and Costo (and Walmart and CVS) are the same thing. (YesVideo “white labels” — secretly does the work — for those other chains.)

Southtree does its own work out of a 40,000 square-foot production facility in downtown Chattanooga, Tennessee. They have over 1,000 playback machines. I liked what I read on its site.

[…]

What’s really odd is that Southtree, YesVideo and other competitors seem hell-bent on converting your old tapes to DVDs. […] What I wanted, of course, was for all of the video to be put onto a hard drive or flash drive, but none of that is available on Southtree’s price list (or YesVideo’s).

Nick assured me, though, that they do offer this service. If that’s what you want, you sign up for the DVD conversion and then type something like, “copy to the drive I’ve supplied” in your order notes.

Friday, June 3, 2016

Twitterrific Adds Patronage

Ged Maheux:

Most importantly, however, today’s release addresses a long-standing request we’ve received from users over the years – a way to donate additional funds towards the continued development of the app. To address this desire we’re pleased to introduce Ollie’s Tip Jar – a series of in-app purchases that users can voluntarily buy to help keep Twitterrific up and running.

The Tip Jar is our way of continuing to offer free and notable updates to Twitterrific while still (hopefully) paying for the cost of development. Since it’s launch in December of 2012, Twitterrific has been updated over 40 times – all for free. Rather than stopping development on Twitterrific version 5 and launching an all-new paid version 6, we’ve decided to include the Tip Jar in the hopes that users who enjoy and love the app will give generously so we can continue to provide updates.

Previously: Overcast 2.5.

Fixing App Store Discovery

John Voorhees:

The App Store’s Top Free, Paid, and Grossing charts are probably the most effective aspect of the App Store […] The trouble is, without a good way to discover apps beyond the most popular ones, it is extremely hard for new apps to break into the charts. Instead, the very existence of the charts perpetuates the position of the apps already on them, making it even more difficult for a new app to break into the charts.

[…]

As I write this, Twitterrific is the fourth result if you search for ‘twitter.’ The search results also include many other relevant results, but still return games, alarm clock apps, and other irrelevant results.

In my own experience, I have found that search results are also often inconsistent between platforms. The results returned for an app search in iTunes are often different than the results I get from the App Store on iOS, which are both different than the results returned by the iTunes Search API, which are oftentimes the worst of all. In short, App Store search results have two main problems – relevance and consistency.

[…]

Another area that needs work is search ranking. Precisely what goes into the ranking of search results is unknown and changes from time to time, but the rankings seem broken, which is curious.

[…]

The App Store's ability to filter search results should also be expanded. This is something that Amazon does particularly well. If I run a search and it returns thousands of results, I can narrow the search by criteria like price, color, and rating.

A2: Analog Malicious Hardware

Kaiyuan Yang et al. (PDF) (via Brendan O’Connor):

While the move to smaller transistors has been a boon for performance it has dramatically increased the cost to fabricate chips using those smaller transistors. This forces the vast majority of chip design companies to trust a third party— often overseas—to fabricate their design. To guard against shipping chips with errors (intentional or otherwise) chip design companies rely on post-fabrication testing. Unfortunately, this type of testing leaves the door open to malicious modifications since attackers can craft attack triggers requiring a sequence of unlikely events, which will never be encountered by even the most diligent tester.

In this paper, we show how a fabrication-time attacker can leverage analog circuits to create a hardware attack that is small (i.e., requires as little as one gate) and stealthy (i.e., requires an unlikely trigger sequence before effecting a chip’s functionality).

Andy Greenberg:

In fact, researchers at the University of Michigan haven’t just imagined that computer security nightmare; they’ve built and proved it works. In a study that won the “best paper” award at last week’s IEEE Symposium on Privacy and Security, they detailed the creation of an insidious, microscopic hardware backdoor proof-of-concept. And they showed that by running a series of seemingly innocuous commands on their minutely sabotaged processor, a hacker could reliably trigger a feature of the chip that gives them full access to the operating system. Most disturbingly, they write, that microscopic hardware backdoor wouldn’t be caught by practically any modern method of hardware security analysis, and could be planted by a single employee of a chip factory.

[…]

The “demonically clever” feature of the Michigan researchers’ backdoor isn’t just its size, or that it’s hidden in hardware rather than software. It’s that it violates the security industry’s most basic assumptions about a chip’s digital functions and how they might be sabotaged. Instead of a mere change to the “digital” properties of a chip—a tweak to the chip’s logical computing functions—the researchers describe their backdoor as an “analog” one: a physical hack that takes advantage of how the actual electricity flowing through the chip’s transistors can be hijacked to trigger an unexpected outcome.

Thursday, June 2, 2016

Sandbox Workaround for Blizzard Apps

Stefan Esser posted a screenshot of code where it looks like Apple’s OSes skip a sandbox check if the code is running under Blizzard’s team identifier. Stefan Esser:

I wonder why the iOS sandbox omits certain sandbox checks if code is signed by Blizzard Entertainment Inc.

It looks like if you are signed by blizzard you can execute whatever executables you find on an iOS device.

Luckily we all know that Blizzard Games never have remote vulnerabilities :-)

This struck people as dangerous and unfair, which perhaps speaks to how much faith developers have in Apple fairly enforcing its own rules. However, it turns out that it’s not actually a sandbox exception but rather a workaround for a crashing bug.

POM:

This path is for the access() call, not for the actual enforcement.

But it doesn’t mean they can execute, it means we make them believe they could.

John Gruber:

Practically speaking, all sandboxing rules still apply to Blizzard apps; workaround doesn’t allow operations that other apps can’t do too.

And Blizzard has fixed their updater, so the workaround shouldn’t be needed in next update.

My takeaway is that Apple will go to extraordinary lengths to avoid crashers in super-popular apps, even when it’s entirely the app’s fault.

The Secret History of Mac Gaming

Richard Moss has Kickstarter-like book proposal (via Hacker News):

The Macintosh changed videogames. It seldom gets credit for this, but it did. It — and its tight-knit community — challenged games to be more than child’s play and quick reflexes. It showed how to make human computer interaction friendly, inviting, and intuitive.

Mac gaming led to much that is now taken for granted by PC gamers, including mouse-driven input, multi-window interfaces, and even online play. The Mac birthed two of the biggest franchises in videogame history, Myst and Halo, and it hosted numerous “firsts” for the medium.

[…]

The Secret History of Mac Gaming is the story of those communities and the game developers who survived and thrived in an ecosystem that was serially ignored by the outside world. It’s a book about people who made games and people who played them — people who, on both counts, followed their hearts first and market trends second. How in spite of everything they had going against them, the people who carried the torch for Mac gaming in the 80s, 90s, and early 2000s showed how clever, quirky, and downright wonderful videogames could be.

The work draws on archive materials as well as 60+ new interviews with key figures from Mac gaming’s past[…]

Yahoo Disclosures National Security Letters

Yahoo (via Hacker News, Slashdot):

As part of our ongoing commitment to transparency, Yahoo is announcing today the public disclosure of three National Security Letters (NSLs) that it received from the Federal Bureau of Investigation (FBI). This marks the first time any company has been able to publicly acknowledge receiving an NSL as a result of the reforms of the USA Freedom Act.

[…]

As explained in our Transparency Report, an NSL is a type of compulsory legal process used by the FBI in national security investigations. The letters Yahoo is making public today were received in April 2013, August 2013, and June 2015, respectively. Yahoo complied with these three NSLs and, to the extent we had the information requested, we disclosed it as authorized by law. Specifically, we produced the name, address, and length of service for each of the accounts identified in two of the NSLs, and no information in response to the third NSL as the specified account did not exist in our system. Each NSL included a nondisclosure provision that prevented Yahoo from previously notifying its users or the public of their existence.

Wednesday, June 1, 2016

Microsoft’s Upgrade Deceptions Are Undermining Windows 10

Paul Thurrott (via Nick Heer):

For months now, I’ve complained about the software giant’s heavy-handed tactics in trying to trick customers into upgrading to Windows 10. But a recent change to the Get Windows 10 advertisement that is forced on Windows 7 and 8.1 users takes things entirely too far. This is indefensible.

[…]

Last week, Microsoft silently changed Get Windows 10 yet again. And this time, it has gone beyond the social engineering scheme that has been fooling people into inadvertently upgrading to Windows 10 for months. This time, it actually changed the behavior of the window that appears so that if you click the “Close” window box, you are actually agreeing to the upgrade. Without you knowing what just happened.

Update (2016-06-01): Olivier Roux:

Yup, happened overnight to one of the attendants of our 2-weeks international meeting last week, couldn’t reboot her laptop, hosed.

Update (2016-06-02): Peter Maurer:

The one thing I don’t get about the ever-changing Windows 10 upgrade dialog is why it has a standard close widget in the first place…

If the user is supposed to make a choice, present the options as buttons, and disable the close widget. Force them to make that choice.

Update (2016-06-03): See also: Hacker News.

Server Side Swift vs. The Other Guys

Qutheory:

It’s clear that something simple like reading and responding JSON could be made better in a lot of the popular web frameworks out there. The ability to access nested request data easily and without worrying about crashes is essential to every day web development. Swift makes some of these basic, repetitive tasks more concise, expressive, and safe. It’s also a lot faster.

Every Default OS X Desktop Picture

Gruber Kristóf (via Nick Heer):

The wallpaper behind the windows is always a very important part of a product branding. Apple is particularly good at choosing the right image. Here is a gallery of default wallpapers of every Mac OS X version.

Swift Type Erasure

Rob Napier:

Once upon a time, when Swift was young, there were a couple of types called SequenceOf and GeneratorOf, and they could type erase stuff. “Type erase?” you may ask. “I thought we loved types.” We do. Don’t worry. Our types aren’t going anywhere. But sometimes we want them to be a little less…precise.

In Swift 2, our little type erasers got a rename and some friends. Now they’re all named “Any”-something. So SequenceOf became AnySequence and GeneratorOf became AnyGenerator and there are a gaggle of indexes and collections from AnyForwardIndex to AnyRandomAccessCollection.

[…]

Not only is the type overwhelming, but it ties us to this particular implementation. […] Clearly we’re leaking too much information about our implementation. What’s the point of reverseZip? Is it to return a Zip2Sequence<...>? No. It’s to return a sequence of tuples. We want a type that means “a sequence of tuples.”

[…]

These “Any” type erasers also aren’t like Any and AnyObject, which are protocols that just “hide” the type. You can still as! an AnyObject back to its original type. AnySequence and its kin completely encapsulate the underlying data. You can’t get the original back. This creates a very strong abstraction layer and strengthens type safety by making as! casting impossible.

[…]

This kind of type eraser lets us convert a protocol with associated types into a generic type. That means we can put it in properties and return values and other places that we can’t use protocols directly.

Samuel E. Giddins:

To start out, we’re going to write a small class that also conforms to SequenceType and is also generic over the Element we want […]

“But wait!” you probably want to say, “that doesn’t really conform to SequenceType!” It does. This is using the same fatalError trick I mentioned earlier, to gloss over the holes in the SequenceType type signatures we can’t possibly know how to fill.

[…]

The most important thing to notice about this class declaration is that _AnySequenceBox is generic over a particular SequenceType implementation. In fact, it’s generic over the very type that our AnySequence wrapper is erasing. It’s also a subclass of _AnySequenceBoxBase, with the base’s generic element being that of the erased type’s Generator.Element. This is the point of translation from our associated type protocol into the world of generics. The rest of the implementation for AnySequence is basically just boilerplate (albeit a vast quantity thereof).

[…]

Yes, we’ve implemented the methods of SequenceType three times. And you’d need to do this again for every protocol you wish to provide a type-erased wrapper for. It’s rather a lot of code to write, but doing this work will make your protocols feel more like first class citizens.

Gwendolyn Weston:

This is where type reification comes in: We make an abstract type concrete by filling in its placeholder types.

[…]

Wrapper classes are conventionally prefixed with the word “Any”, in order to guarantee that you will instantiate an object that implements our protocol and fills the generic type, without necessarily having the implementation on hand.

[…]

I feel the best definition that I have at the end of my road of preparing this talk is that type erasure is this design pattern where you have a wrapper class that has a constraint in the initializer method such that you can only initialize it with an instance of an implementation of the protocol you are trying to make concrete.

Russ Bishop:

More specifically, what is type erasure in the context of Swift and Protocols with associated types? If I could boil it down to one sentence:

Passing a generic type parameter to a protocol’s associated type member.

Natasha Murashev:

This still feels like an overly complex fix for something that should just work and hopefully it will in the future. For now, this is a super good pattern to know when working with protocols with associated types.

See also: Type erasure in C++: how boost::shared_ptr and boost::function work.

Previously: Swift Protocols With Associated Types.

Update (2016-06-01): David Owens II:

If you have to keep using type erasures to actually work with your types, doesn’t that suggest something? Maybe I don’t get it.

Joe Groff:

Being only as precise as you need to be lets you stay flexible and avoid unnecessary refactoring.

Only keep the types you need. Type hoarding is unhealthy and may require professional help.

Update (2016-06-02): Elmar Kretzer posted an example of a type-erasing struct.

Update (2018-03-09): See also: Hector Matos and Mike Ash (Hacker News).

Copyright © 2000–2024 Michael Tsai.