Thursday, February 23, 2017 [Tweets] [Favorites]

SHA-1 Collision

Google (Hacker News):

Today, 10 years after of SHA-1 was first introduced, we are announcing the first practical technique for generating a collision. This represents the culmination of two years of research that sprung from a collaboration between the CWI Institute in Amsterdam and Google. We’ve summarized how we went about generating a collision below. As a proof of the attack, we are releasing two PDFs that have identical SHA-1 hashes but different content.

For the tech community, our findings emphasize the necessity of sunsetting SHA-1 usage. Google has advocated the deprecation of SHA-1 for many years, particularly when it comes to signing TLS certificates. As early as 2014, the Chrome team announced that they would gradually phase out using SHA-1. We hope our practical attack on SHA-1 will cement that the protocol should no longer be considered secure.


This attack required over 9,223,372,036,854,775,808 SHA1 computations. This took the equivalent processing power as 6,500 years of single-CPU computations and 110 years of single-GPU computations.


The SHAttered attack is 100,000 faster than the brute force attack that relies on the birthday paradox. The brute force attack would require 12,000,000 GPU years to complete, and it is therefore impractical.


Basically, each PDF contains a single large (421,385-byte) JPG image, followed by a few PDF commands to display the JPG. The collision lives entirely in the JPG data - the PDF format is merely incidental here. Extracting out the two images shows two JPG files with different contents (but different SHA-1 hashes since the necessary prefix is missing). Each PDF consists of a common prefix (which contains the PDF header, JPG stream descriptor and some JPG headers), and a common suffix (containing image data and PDF display commands).

The header of each JPG contains a comment field, aligned such that the 16-bit length value of the field lies in the collision zone. Thus, when the collision is generated, one of the PDFs will have a longer comment field than the other. After that, they concatenate two complete JPG image streams with different image content - File 1 sees the first image stream and File 2 sees the second image stream. This is achieved by using misalignment of the comment fields to cause the first image stream to appear as a comment in File 2 (more specifically, as a sequence of comments, in order to avoid overflowing the 16-bit comment length field). Since JPGs terminate at the end-of-file (FFD9) marker, the second image stream isn’t even examined in File 1 (whereas that marker is just inside a comment in File 2).

I think SHAttered overstates the impact on Git. Linus Torvalds (2005, via Joe Groff):

I really hate theoretical discussions.

The fact is, a lot of crap engineering gets done because of the question ”what if?”. It results in over-engineering, often to the point where the end result is quite a lot measurably worse than the sane results.

You are literally arguing for the equivalent of “what if a meteorite hit my plane while it was in flight - maybe I should add three inches of high-tension armored steel around the plane, so that my passengers would be protected”.


And the thing is, if somebody finds a way to make sha1 act as just a complex parity bit, and comes up with generating a clashing object that actually makes sense, then going to sha256 is likely pointless too - I think the algorithm is basically the same, just with more bits. If you’ve broken sha1 to the point where it’s that breakable, then you’ve likely broken sha256 too.

He’s being criticized for saying this, but (so far) it looks like he was actually right.

Linus Torvalds:

Put another way: I doubt the sky is falling for git as a sourcecontrol management tool. Do we want to migrate to another hash? Yes. Is it “game over” for SHA1 like people want to say? Probably not.

I haven’t seen the attack details, but I bet

(a) the fact that we have a separate size encoding makes it much harder to do on git objects in the first place

(b) we can probably easily add some extra sanity checks to the opaque data we do have, to make it much harder to do the hiding of random data that these attacks pretty much always depend on.

Previously: MD5 Collision.

Migrating Firefox for iOS to Swift 3.0

Mozilla (via Emily Toop):

A week ago we completed the migration of the entire Firefox for iOS project from Swift 2.3 to Swift 3.0. With over 206,000 lines of code, migrating a project of this size is no small feat. Xcode’s built in conversion tool is a fantastic help, but leaves your codebase in a completely uncompilable state that takes a good long while to resolve.


The first hitch in the plan occurred fairly quickly. Our test targets, despite not importing code from other targets further down the dependency tree, all required our primary target, Client, as the host app in order to run. Therefore our plan to ensure each target robustly passed its tests before moving onto the next target was impossible. We would have to migrate all of the targets, then the test targets and then ensure that the tests pass. This would mean that we may possibly be performing code changes in dependent targets on incorrectly migrated code, which added an extra layer of uncertainty. In addition, being unable to execute the code before moving on would mean that if we made a poor decision when solving a migration issue, that decision may end up proliferating through many targets before we realised that the code change produces a crash.

The second hitch came when migrating some of the larger targets, in particular Storage. Even after all this time, Xcode’s ability to successfully compile Swift is…flaky. When, after performing the auto-conversion, your first build error is a segfault in Xcode, this is not at all helpful. Especially when the line of code mentioned in the segfault stack trace is in an unassuming class that is doing nothing special. And when you comment out all of the code in that class, it still produces a segfault.


It had taken 3.5 engineers, 3 members of QA and 3.5 weeks, but the feeling when we were finally ready to hit merge was jubilant.

They ran into an interesting NSKeyedArchiver issue.

Previously: Getting to Swift 3 at Airbnb.

MagicGrips for Magic Mouse

Chance Miller:

Elevation Lab, the company behind a handful of popular Apple accessories, is today announcing its latest product: MagicGrips. The company says that this accessory is designed to work with the Magic Mouse and makes it easier to grip.

Elevation Lab says that this accessory makes the Magic Mouse more comfortable use by widening your grip and allowing you to squeeze the mouse without it moving upwards. Furthermore, the grip is said to release hand tension.

The Magic Mouse is the least comfortable mouse I have ever used, due to the shape of the edges. This looks like it would help.

(Anyone remember the name of the case that snapped around the original iMac puck mouse to give a more standard shape?)

Previously: Apple’s New Magic Keyboard, Mouse, and Trackpad, Magic Mouse Review.

Update (2017-02-23): I think the iMac mouse adapter that I used was the UniTrap.

Opening the User Library Folder

Rob Griffiths:

Yesterday, I wrote about an apparent change in Finder’s Library shortcut key. To wit, it used to be that holding the Option key down would reveal a Library entry in Finder’s Go menu.

However, on my iMac and rMBP running macOS 10.12.3—and on others’ Macs, as my report was based on similar findings by Michael Tsai and Kirk McElhearn—the Option key no longer worked; it was the Shift key. But on a third Mac here, running the 10.12.4 beta, the shortcut was back to the Option key.


After some experimentation, I was able to discover why the shortcut key changes, and how to change it between Shift and Option at any time. This clearly isn’t a feature, so I guess it’s a bug, but it’s a weird bug.

Video Pros Moving From Mac to Windows for High-End GPUs

Marco Solorio (May 2016):

But as good as that juiced up Mac Pro Tower is today, I know at some point, the time will have to come to an end, simply because Apple hasn’t built a PCIe-based system in many years now. As my article described, the alternative Mac Pro trashcan is simply not a solution for our needs, imposing too many limitations combined with a very high price tag.

The Nvidia GTX 1080 might be the final nail in the coffin. I can guarantee at this point, we will have to move to a Windows-based workstation for our main edit suite and one that supports multiple PCIe slots specifically for the GTX 1080 (I’ll most likely get two 1080s that that new price-point).


Even a Thunderbolt-connected PCIe expansion chassis to a Mac Pro trashcan wont help, due to the inherent bandwidth limits that Thunderbolt has as compared to the buss speeds of these GPU cards. And forget about stacking these cards in an expansion chassis… just not going to happen.

Via John Gruber:

This may be a small market, but it’s a lucrative one. Seems shortsighted for Apple to cede it.

Timo Hetzel:

Moving my video workflow to a modern PC could save me an estimated 4-8 hours every week. I wonder if Apple knows/cares.

Previously: Getting a New 2013 Mac Pro in 2017, How Apple Alienated Mac Loyalists.

Wednesday, February 22, 2017 [Tweets] [Favorites]

Overcast 3

Marco Arment (tweet):

Previously, tapping an episode in the list would immediately begin playback. This is nice when you want it, but accidental input was always an issue: I found it too easy to accidentally begin playing something that I was trying to rearrange, delete, or see info about.


Some kind of “Up Next”-style fast queue management has been one of Overcast’s most-requested features since day one. It took me a long time to come around to the idea because I thought my playlists served the same role.


Google provides an extensive control panel that lets you block certain ad categories. Most are clearly placed in Sensitive Categories and were easily disabled before launch, like gambling, drugs, etc., but I kept hearing from customers who’d seen other ads that offended both of us.


No closed-source code will be embedded in Overcast anymore, and I won’t use any more third-party analytics services. I’m fairly confident that Apple has my back if a government pressures them to violate their customers’ rights and privacy, but it’s wise to minimize the number of companies that I’m making that assumption about.

Fortunately, the Google ads made relatively little — about 90% of Overcast’s revenue still comes from paid subscriptions, which are doing better now. The presence of ads for non-subscribers is currently more important than the ads themselves, so I can replace them with pretty much anything. So I rolled my own tasteful in-house ads with class-leading privacy, which show in the Now Playing and Add Podcast screens[…]

I really like the interface refinements in this version. I didn’t use episode playlists much before because it was so awkward to add to them. Now it’s easy.

Triaging episodes is much easier now, too. It’s easier to read the summary, I don’t accidentally play the episode when I just wanted to see its info, and no more swipe-to-delete, which had been slow and not fully reliable.

I think it still needs some work for handling unfinished episodes, though. If I don’t add them to a playlist before starting them, it’s too easy to switch to another episode, lose my place, and forget to go back. I would like to see either a history view or the return of the In Progress smart playlist.

See also: Steven Aquino, John Gruber, Federico Viticci (tweet).

Previously: Twitter Sells Fabric to Google.

Update (2017-02-23): Jason Snell:

Now, with Overcast 3, I have a different approach. I now have two playlists. One, called Priority Playlist, basically functions as my play queue. That’s the stuff I will definitely listen to if I have the time, ordered in a way to keep me happy. A few of my must-listen podcasts add their episodes to this playlist automatically, but most don’t.

Setting my playlist settings (left) and adding an item to my playlist (right). The second playlist is called All Episodes, and as the name would imply, it shows every podcast episode from every podcast I subscribe to, with the newest episodes at the top. From this list, I can scroll to see what’s new and if anything pops up as an immediate must-listen. When I find such an episode, I tap once to reveal Overcast 3’s new episode-action strip, tap the Add icon, and then tap “Add to Priority Playlist” or, if I’m really excited, “Play Next.”

Google Site Search Discontinued

Barb Darrow (Hacker News):

This spring, Google plans to discontinue Google Site Search, a product it has sold to web publishers that wanted to apply the industry’s leading search technology to their own sites.


Once a customer’s allocation of search queries is exhausted, the account will “automatically convert” to the company’s Custom Search Engine, or CSE for short.


CSE is a free, advertising-supported version of Google’s search technology, that provides similar features and functions to GSS, according to the email.

This is disappointing. The e-mail that I received seemed to suggest that I should look into Google Cloud Search, but that’s a totally different product. To provide a search engine for my Web site, I would need to switch to CSE. Years ago, I switched from CSE to GSS because I wanted a better user experience and no ads. CSE devotes much more of the page to ads than a regular Google search; on my 30-inch display, the actual search results from my site start more than halfway down. Now, Google apparently would rather show ads than let me keep paying for GSS.

I’m not sure yet what I’ll do. I have been using DuckDuckGo’s search for this blog, but when I tried it on the C-Command site the results were much worse (less relevant and incomplete) than Google’s. However, that was a while ago, so perhaps it’s better now.

See also: Barry Schwartz.

Update (2017-02-22): There are also changes to CSE.

OmniOutliner Essentials

Ken Case (tweet):

In OmniOutliner’s new Essentials edition, your entire focus is on your own content: there are no distracting sidebars or panels. You can choose to work in a window or in a distraction-free full-screen mode, selecting from a set of beautiful built-in themes. As you write, you’ll be able to see some key statistics about your content so you can track progress towards your goals. But our goal is to help you focus on your content and whatever task you’re working on—not on the tool you’re using.

With the Essentials edition, we’ve lowered OmniOutliner’s entry price from $49.99 to an extremely affordable $9.99.

Brent Simmons:

MORE was by Living Videotext, which was Dave Winer’s company. Later I went to work at Dave’s company UserLand Software, which also included an outliner in its app Frontier, which I worked on. So there is a sort-of family tree connection from OmniOutliner back to MORE.

Thunderbolt 3 and USB-C Infographic

Lloyd Chambers:

The infographic from OWC shown below might help in some ways, but there are various “gotchas”. MPG recommends generally buying full-speed Thunderbolt 3 cables, for maximum interoperability. However, lower speed cables intended for use with USB-C have their place also.

Monday, February 20, 2017 [Tweets] [Favorites]

Provide Custom Collections for Dictionary Keys and Values


This proposal address two problems:

  • While a dictionary’s keys collection is fine for iteration, its implementation is inefficient when looking up a specific key, because LazyMapCollection doesn’t know how to forward lookups to the underlying dictionary storage.
  • Dictionaries do not offer value-mutating APIs. The mutating key-based subscript wraps values in an Optional. This prevents types with copy-on-write optimizations from recognizing they are singly referenced.


Dictionary values can be modified through the keyed subscript by direct reassignment or by using optional chaining. Both of these statements append 1 to the array stored by the key "one":

// Direct re-assignment
dict["one"] = (dict["one"] ?? []) + [1]

// Optional chaining

Both approaches present problems.

The proposed solution is an improvement but still seems a bit awkward.

Update (2017-02-20): Airspeed Velocity:

There are other changes likely for Swift 4 that would make this less awkward (e.g. to supply a default value when subscripting).

It’s just that this particular change was ABI impacting (changes the type of Dictionary.Value) so was proposed during stage 1.

Dash 4

Kapeli (tweet):

Docset Playgrounds – Most docsets now show “Play” buttons which let you quickly test snippets of code


Search Using Selected Text – This feature has been completely remade and is now more reliable, even in apps that don’t support system services.


Tab Improvements – You can now reopen the last closed tab, duplicate tabs and close all tabs except for the selected one


You can now copy the external URL of documentation pages for easier sharing

To make the Play button show up for Apple’s documentation, make sure that the language is set to Swift.

Ruby’s reject!

Accidentally Quadratic (Hacker News):

The code used to be linear, but it regressed in response to bug #2545, which concerned the behavior when the block passed to reject! executed a break or otherwise exited early. Because reject! is in-place, any partial modifications it makes are still visible after an early exit, and reject! was leaving the array in a nonsensical state. The obvious fix was to ensure that the array was always in a consistent state, which is what resulted in the “delete every time” behavior.

I find this interesting as a cautionary tale of how several of Ruby’s features (here, ubiquitous mutability, blocks, and nonlocal exits) interact to create suprising edge cases that need to be addressed, and how addressing those edge cases can easily result in yet more problems (here, quadratic performance).

Saturday, February 18, 2017 [Tweets] [Favorites]

The State of iBooks in Early 2017

Michael E. Cohen:

iBooks is not quite as unreliable and confusing as it was when I wrote about it last year, but neither has it improved nearly as much as loyal iBooks users deserve. Moreover, what little support documentation Apple provides is sketchy and inaccurate, leaving the impression that even the support and documentation departments within Apple are ignoring iBooks.


Now and then, the Library window in iBooks on the Mac gets confused, showing books that aren’t there, or duplicating thumbnails for books that are there. A couple of days ago, I found two thumbnails for a short EPUB I had just uploaded. The second thumbnail was a phantom, and actually represented the next book in the window, which I only discovered by right-clicking the duplicate. A simple cosmetic problem you might think, but if I had attempted to delete the phantom copy, I would have trashed an entirely different book!

Previously: I Wish Apple Loved Books.

Swift Ownership Manifesto

Apple (mailing list):

The widespread use of copy-on-write value types in Swift has generally been a success. It does, however, come with some drawbacks:

  • Reference counting and uniqueness testing do impose some overhead.

  • Reference counting provides deterministic performance in most cases, but that performance can still be complex to analyze and predict.

  • The ability to copy a value at any time and thus “escape” it forces the underlying buffers to generally be heap-allocated. Stack allocation is far more efficient but does require some ability to prevent, or at least recognize, attempts to escape the value.

Certain kinds of low-level programming require stricter performance guarantees. Often these guarantees are less about absolute performance than predictable performance. For example, keeping up with an audio stream is not a taxing job for a modern processor, even with significant per-sample overheads, but any sort of unexpected hiccup is immediately noticeable by users.

Another common programming task is to optimize existing code when something about it falls short of a performance target. Often this means finding ”hot spots” in execution time or memory use and trying to fix them in some way. When those hot spots are due to implicit copies, Swift’s current tools for fixing the problem are relatively poor; for example, a programmer can fall back on using unsafe pointers, but this loses a lot of the safety benefits and expressivity advantages of the library collection types.

We believe that these problems can be addressed with an opt-in set of features that we collectively call ownership.

Previously: Chris Lattner ATP Interview, Swift Plans.

Dart-C USB-C Laptop Charger

David Pogue:

The Dart-C, billed as the world’s smallest laptop charger. And it really is tiny.

Yet somehow, it provides 65 watts—plenty for laptops like the 12- and 13-inch MacBooks, the Lenovo ThinkPad 13, ASUS ZenBook 3, Dell XPS 13, and so on. Really honking laptops, like the 15-inch MacBook Pro, expect more wattage (85). This charger will work on those machines—just not as fast.

How do I love this thing? Let us count the ways.

  • It has a standard USB jack embedded in the cable. That means that you can simultaneously charge your phone, tablet, camera, or whatever—with no slowdown in charging your primary gadget.
  • It has an indicator light that lets you know if you’re plugged into a working outlet. (Apple’s chargers no longer have a status light.)

It’s even more expensive than Apple’s charger, though.

Fixing (and Explaining) PDFpen 8.3.1’s Crash on Launch

Adam C. Engst:

Greg said that the reason PDFpen crashed — even before it actually launched — was because Smile’s developer signing certificate from Apple had expired.


In the past, the expiration of a code signing certificate had no effect on already shipped software. PDFpen 6.3.2, which Smile still makes available for customers using OS X 10.7 Lion, 10.8 Mountain Lion, and 10.9 Mavericks, is signed with a certificate that expired long ago, and it has no trouble launching.

What’s new with PDFpen 8 is that, in addition to being code signed, it has a provisioning profile, which is essentially a permission slip from Apple that’s checked against an online database in order to allow the app to perform certain actions, called entitlements. For PDFpen, the entitlement that’s being granted is the capability to access iCloud despite being sold directly, rather than through the Mac App Store, a feature that wasn’t possible until about a year ago.

It sounds like every Developer ID app that uses iCloud has a built-in time bomb. Something is not designed properly here. First, why does Apple issue certificates with relatively short expiration dates? They already have a means of revoking certificates in the event of a problem. Second, why does the OS check whether the code signing certificate is valid now, as opposed to when the provisioning profile was signed?

Previously: CloudKit and Map Kit for Gatekeeper Apps, More Mac App Store Certificate Problems, WWDR Intermediate Certificate Expiration.

Update (2017-02-20): See also: MacRumors, Acqualia.

Update (2017-02-22): Rick Fillion:

Due to the expired Provisioning Profile, 1Password mini wouldn’t launch. And without mini running, 1Password itself was unable to startup successfully. Both mini and 1Password itself were signed with the same Developer ID certificate. Gatekeeper allowed 1Password to run, but due to the different rules for apps with provisioning profiles, it would not allow mini to run.

As far as we can tell, the only way to correct this problem is to provide a new build of the app with an updated provisioning profile with a new expiration date.


When we generated our updated provisioning profile we also needed to generate a new Developer ID certificate. We didn’t realize it at the time, but the common name of newly created certificates now include the team identifier in addition to the company name; “Developer ID Application: AgileBits Inc. (2BUA8C4S2C)” vs. “Developer ID Application: AgileBits Inc.”. Close. Super close. But we weren’t looking for a “close” match.

Rick Fillion:

In case you’re wondering how to tell when a provisioning profile will expire you can run security cms -D -i on the Terminal to have it output information about a profile.

Thursday, February 16, 2017 [Tweets] [Favorites]

Deferring ABI Stability From Swift 4

Ted Kremenek (tweet, Reddit):

Given the importance of getting the core ABI and the related fundamentals correct, we are going to defer the declaration of ABI stability out of Swift 4 while still focusing the majority of effort to get to the point where the ABI can be declared stable.

To allow the community to follow along with this effort, an ABI dashboard will get wired up from the swift-evolution home page that will present a table of main ABI tasks remaining and what Swift release they landed in.


The Swift 4 compiler will provide a source-compatibility mode to allow existing Swift 3 sources to compile, but source-breaking changes can manifest in “Swift 4” mode. That said, changes to fundamental parts of Swift’s syntax or standard library APIs that breaks source code are better front-loaded into Swift 4 than delayed until later releases. Relative to Swift 3, the bar for such changes is significantly higher[…]

Swift 4 Release Process:

Swift 4 is a major release that is intended to be completed in the fall of 2017. It pivots around providing source stability for Swift 3 code while implementing essential feature work needed to achieve binary stability in the language. It will contain significant enhancements to the core language and Standard Library, especially in the generics system and a revamp of the String type.


The intended design is that a project containing multiple Swift modules, such as an Xcode project with multiple Swift targets, will be able to adopt into the specific Swift language mode on a per module (target) level and that they can freely interact within the same compiled binary. Note that this interoperability only exists at the binary level when the targets are compiled with the same compiler.

Postponing ABI stability (again) makes sense, especially given that String is still in flux.

Previously: Chris Lattner ATP Interview, Swift 4 String Manifesto, ABI Stability Deferred Until After Swift 3.0, Looking Back on Swift 3 and Ahead to Swift 4.

Being a Mutable Collection is not Sufficient to be a MutableCollection

Ole Begemann:

A MutableCollection supports in-place element mutation. The single new API requirement it adds to Collection is that the subscript now must also have a setter.


MutableCollection allows changing the values of a collection’s elements, but the protocolʼs documentation stipulates that the mutation must neither change the length of the collection nor the order of the elements. Set canʼt satisfy either of these requirements.


All Dictionary would gain from conforming to MutableCollection and/or RangeReplaceableCollection would be methods that operate on Index values and (Key, Value) pairs, which is probably not compelling enough to invest anything in the conformance even if it were compatible with the typeʼs implementation.

Ole Begemann:

CharacterView does conform to RangeReplaceableCollection but not to MutableCollection. Why? A string is clearly mutable; it seems logical that it should adopt this protocol. Again, we need to consider the protocolʼs semantics.


However, the Characterʼs size in the underlying storage is not the same for all characters, so replacing a single Character can potentially make it necessary to move the subsequent text forward or backward in memory by a few bytes to make room for the replacement. This would make the simple subscript assignment potentially an O(n) operation, and subscripting is supposed to be O(1).


The final potential issue for CharacterViewʼs hypothetical MutableCollection conformance is Unicode and the complexities it brings. The existence of combining characters means that replacing a single Character can actually change the stringʼs length (measured in Characters) if the new character combines with its preceding character.

Previously: Swift 4 String Manifesto.

Apple Fighting New “Right to Repair” Legislation

Ben Lovejoy (Hacker News):

Apple is fighting ‘right to repair’ legislation which would give consumers and third-party repair shops the legal right to purchase spare parts and access service manuals. The state of Nebraska is holding a hearing on the proposed legislation next month, and Motherboard reports that Apple will be formally opposing the bill.

[…] hopes that getting a single state to pass a right to repair bill will result in manufacturers giving in, citing the precedent of similar legislation in the car industry.

Previously: Error 53.

Great Alternatives to Hamburger Menus

Kara Pernice and Raluca Budiu:

Hidden navigation, such as the hamburger menu, is one of the many patterns inspired by mobile designs. Screen space is a precious commodity on mobile. To meet the challenge of prioritizing content while still making navigation (and other chrome) accessible, designers commonly rely on hiding the navigation under a menu — often indicated by the infamous hamburger icon. Like a cheap fast food chain, it got designers addicted to its convenience, and now serves millions each day, both on mobile devices and on desktops.

While our qualitative user testing has repeatedly shown that navigation hidden under a drop-down menu is less discoverable on the desktop, we wanted to measure the size of this effect in a quantitative study and assess the relative impact of hidden navigation on the desktop versus mobile.

Mobiscroll (via Andy Bargh):

Hamburger menus drive engagement down, slow down exploration and confuse people. If you are reading this, it won’t confuse you, but it damn will confuse others who might be happy to consume your content.


I cannot stress this enough. Always design with real content, otherwise you’ll end up with placeholders, lorem ipsums and hamburger menus inside hamburger menus. Content on its own doesn’t make sense, and layouts without content either.

Previously: Apple on Hamburger Menus, The Hamburger Menu Doesn’t Work, Ex-Microsoft Designer Explains the Move Away From Metro, Hamburgers and Basements.

Kindle Direct Paperbacks

Amazon (via Matt Henderson):

We’re excited to offer the opportunity to publish paperbacks in addition to Kindle eBooks. We’ll be adding even more print-related features in the future, like proof copies, author (wholesale) copies, and expanded distribution to bookstores and non-Amazon websites. CreateSpace still offers these features, and KDP will offer them as well.

Publishing a paperback can help you reach new readers. KDP prints your book on demand and subtracts your printing costs from your royalties, so you don’t have to pay any costs upfront or carry any inventory.

What Happened With the Apple TV 4

Mark Gurman (tweet):

Apple doesn’t disclose how many Apple TVs it sells, but Chief Financial Officer Luca Maestri acknowledged in a recent interview that sales decreased year-over-year from the 2015 holiday season to this past 2016 holiday period. The research firm eMarketer says the fourth-generation Apple TV has steadily lost market share since its release in the fall of 2015; in January just 11.9 percent of connected television customers were using it, the research firm says, down from 12.5 percent in September. In part, the slide reflects competition from Amazon and Roku, whose boxes do the same and more for less money.


Apple had a backup plan if it wasn’t able to replace the existing cable box—the much-ballyhooed “skinny bundle,” a stripped down web service that would let viewers choose channels rather than paying for ones they don’t watch. Apple proposed bundling the four main broadcast networks and a handful of cable channels as well as on-demand TV shows and movies for $30 to $40 a month. The media companies were willing to engage with Apple due to concerns about the rise of online services like Netflix and the cord-cutting phenomenon.

But the two sides stumbled over cost, the composition of the bundles and negotiating tactics. The media companies blamed Apple’s arrogance; Apple blamed the media companies’ inflexibility. In the end, the talks fell apart, leaving Apple to tout stripped-down bundles from Sony PlayStation and DirecTV. After the negotiations foundered, Apple’s hardware team ditched the coaxial port.

Update (2017-02-18): Dan Moren:

It’s tough to figure out what that extra cash buys you, beyond the Apple brand. The previous version of the Apple TV, which was less powerful though still perfectly capable at its primary function of streaming video, cost a more comparable $99. Frankly, though I use the Apple TV every day, I’d be hard-pressed to find $50 more worth of functionality on the new model. (Most of that cost increase is probably attributable to the Siri remote, which I have mostly ditched in favor of a harder-to-lose and more user-friendly Logitech Harmony universal remote.)


We talk a lot about the “second screen” experience of people sitting in front of the TV: how they’re checking Twitter or looking actors up on IMDb or Wikipedia while watching. But when it comes to apps, it’s not the iPhone or iPad or MacBook that’s the second screen—it’s the Apple TV.

Wednesday, February 15, 2017 [Tweets] [Favorites]

Grand Central Dispatch’s Achilles Heel

Wil Shipley (tweet):

I don’t know much about the internals of GCD so I can’t speak with authority, but it seems like this could be solved with a couple of minor changes to sync(): figure out if the destination queue is the current queue, and if so just execute the submitted block immediately and return. This wouldn’t even be a source or binary-breaking change, because, again, the current behavior is HANG the app.

And, in fact, this is the workaround third-party programmers have made for the last several years. If you do a Google search for dispatch_get_current_queue [now deprecated] you’ll see a bunch of developers complaining about that call disappearing because they were using it for this hack.


Hopefully you’re as horrified by this mess as I am. This is the very model of spaghetti code. Last week I ported this file from macOS 10.8 to 10.12 and honestly I still couldn’t come up with a good way to re-architect it. I’m bending over backwards to interact with the main thread in multiple places in this codebase and I’m not sure if I’m on the main thread or not and it’s a nightmare.

Greg Parker:

dispatch_get_current_queue() == someQueue is insufficient to avoid the deadlocks you describe.

Rob Napier:

Interestingly, they did fix this in CoreData on top of GCD. performAndWait is reentrant. But unsure how they implemented.

Reverse Engineering

Alex Denisov:

The task becomes easy since the six colors are hardcoded in the binary. I just need to find where exactly and change the values to ones I like more.


What is important here: three consecutive values starting at 0x10000c790 moved to xmm_ registers. I’m not 100% sure, but I’d assume that they are used to pass parameters into colorWithDeviceRed:green:blue:alpha:.


Now I know where the colors reside in code. I need to find them in the binary. The address of a first color component is 0x10000c790. To find its on-disk address I need to subtract a base address from it. The base address can be obtained via LLDB as well.


Now I can use xxd with -s (--seek) and -l (--length) parameters to get exactly 8 bytes at a given address.

Switch 2FA From SMS to an App

Laura Shin (via David Heinemeier Hansson):

“So I called the company to make sure I hadn’t forgotten to pay my phone bill, and they said, you don’t have a phone with us. You transferred your phone away to another company,” he says. A hacker had faked his identity and transferred his phone number from T-Mobile to a carrier called Bandwidth that was linked to a Google Voice account in the hacker’s possession. Once all the calls and messages to Kenna’s number were being routed to them, the hacker(s) then reset the passwords for Kenna’s email addresses by having the SMS codes sent to them (or, technically, to Kenna’s number, newly in their possession). Within seven minutes of being locked out of his first account, Kenna was shut out of of up to 30 others, including two banks, PayPal, two bitcoin services — and, crucially, his Windows account, which was the key to his PC.


Last summer, the National Institutes of Standards and Technology, which sets security standards for the federal government, “deprecated” or indicated it would likely remove support for 2FA via SMS for security. While the security level for the private sector is different from that of the government, Paul Grassi, NIST senior standards and technology advisor, says SMS “never really proved possession of a phone because you can forward your text messages or get them on email or on your Verizon website with just a password. It really wasn’t proving that second factor.”

Usher Will Be Stepping Aside

Many Tricks:

QuickTime is very old, and obviously no longer updated. (It’s so old that it’s not even 64-bit code.) Newer video formats may cause issues, and we can’t resolve those issues in Usher because they’re actually in QuickTime. Given these age-related issues with QuickTime, we’re no longer comfortable selling and supporting Usher to new buyers, so we’ve decided it’s retirement time.


Beyond the market size, we can’t just delete “old QuickTime” and insert “QuickTime X” and be done with it. The two are very different, so much so that we’d need to totally rewrite the engine that drives Usher. And that’s a huge job…and one that wouldn’t ever be paid back in sales, due to the limited market size.

Previously: The Curious Case of QuickTime X, What Is Apple Doing With QuickTime?, AV Foundation and the Void.

Swift and Objective-C Forever?

Jeff Johnson (Hacker News):

When Swift became public in 2014, its creator Chris Lattner seemed to claim that Swift and Objective-C would coexist indefinitely.


The problem is that nobody believes this. And of course Lattner has now left Apple, so he won’t be there to take the criticism is his claim turns out to be false. The consensus among developers is that Apple will eventually deprecate Objective-C, and Swift will become the sole first class language for Cocoa app development.


You only have to review the history of Apple developer relations to see the long string of deprecations, disappointments, suffering, and broken promises. Objective-C garbage collection, 64-bit Carbon, the Cocoa-Java bridge, Yellow Box for Windows, Dylan. Need I go on? I could go on. Apple evangelists will tell you that Swift is the best programming language ever and then turn around and tell you that we’ve always been at war with Swift.


The people who think Apple will deprecate Objective-C, how do they think Apple can handle it? Some people suggest that Apple will deprecate Objective-C externally, but they will continue Objective-C development internally and indefinitely. However, I think these people underestimate the problem. Given the amount of Objective-C code Apple has, and the constraints they’re working under, taking the slow road internally to a Swift future would be a very slow road indeed.

I don’t see Apple dropping Swift. The technology seems to be sound, and Apple has really put its reputation on the line in a way that it didn’t with the other canceled projects. Plus, unlike those, Swift already has massive adoption outside of Apple.

That said, I expect that Objective-C, while not being deprecated, will decline in popularity the same way that Carbon—also officially a first-class coexister—did. I don’t think that Apple will maintain sample code parity for much longer.

Johnson is right that there are many open questions about how Apple will manage this transition—if that’s what it is—both internally and externally. It will affect the OS itself, Apple’s apps, the public APIs, and Apple’s own staffing. It’s got to be difficult for Apple to hire and retain WebObjects programers for its internal services, and it could face similar issues, to a lesser extent, if developers continue to switch to Swift.

Tuesday, February 14, 2017 [Tweets] [Favorites]

On the Uselessness of Search in macOS Mail

Rob Griffiths:

For the last couple macOS releases, I’ve had nothing but trouble searching in Mail. Note that I didn’t write “trouble searching mail,” but rather, “trouble searching in Mail.”

Searching Spotlight directly works for him.

Dan Frakes:

Is there a secret to getting macOS Mail’s “Unread” smart mailbox to show 0 messages when there are no unread messages?

I see both of these issues all the time, though with the Flagged smart mailbox rather than Unread. The number next to the Flagged mailbox is different from the number of messages shown in that mailbox, and some of the messages shown in it are not actually flagged.

As a quick fix to prevent it from showing messages that are not actually flagged, I was able to use this Terminal command:

sqlite3 ~/Library/Mail/V4/MailData/Envelope\ Index 'SELECT * FROM subjects WHERE rowid IN (SELECT subject FROM messages WHERE flagged="1")'

After confirming that these really are messages that should not be flagged, I could mark the messages as unflagged:

sqlite3 ~/Library/Mail/V4/MailData/Envelope\ Index 'UPDATE messages SET flagged="0" WHERE flagged="1"'

(Caveat: I do not use multi-colored flags and have not investigated how this affects them.)

This would only provide temporary relief, though. Rebuilding Mail’s database takes longer but also lasts longer. But because of this and other problems that wouldn’t fully go away, I did a clean reinstall of macOS about a week ago. So far so good, but I suspect that both problems will be back.

Fortunately, most of the mail that I need to search is in FogBugz and EagleFiler.

Update (2017-02-15): Rob Griffiths:

If I move all the messages from an inbox or local storage folder into a different local storage folder, they’ll be indexed and findable. I can then move them back into the inbox or source folder, and they remain findable.

Twitterrific for Mac Kickstarter

Sean Heber (MacRumors):

After much consideration, we decided that the best way forward was to go back to the beginning. Rather than bending the long-neglected Twitterrific for Mac into a new shape, we will borrow what we can from iOS and use it to build a modern new macOS app.


We’re confident that we can do this, but we need your help! Please check out our Kickstarter page, watch the video and study the plan. There are many different funding levels including regular access to beta builds all through Phoenix’s lifespan. If you’re the kind of person who loves to see software evolve through it’s development, or just want to start using a new Twitterrific on your desktop sooner rather than later, this one is for you.

This is our first Kickstarter project and a new way for us to fund our software development. The main reason the Mac app languished is because we aren’t sure that there’s a market for a desktop social networking product (it’s easy to make a case that all our social activities have moved to mobile.) For our small software company, the risk of recouping development costs was just too high. Kickstarter removes this unpredictability and gives us an exact budget to work against.

Jason Snell:

Ten years later, my view of Twitter as a service is still largely framed by apps, rather than the web. If Twitter was only on the Web, I think I’d use it about as often as Facebook, which is to say, not often.

Instapaper Outage Cause & Recovery

Brian Donohue:

Without knowledge of the pre-April 2014 file size limit, it was difficult to foresee and prevent this issue. As far as we can tell, there’s no information in the RDS console in the form of monitoring, alerts or logging that would have let us know we were approaching the 2TB file size limit, or that we were subject to it in the first place. Even now, there’s nothing to indicate that our hosted database has a critical issue.

If we ever had knowledge of the file size limit, it likely left with the 2013-era betaworks contractors that performed the Softlayer migration.


We didn’t have a good disaster recovery plan in the event our MySQL instance failed with a critical filesystem issue that all of our backups were also subject to.


When it became clear the dump would take far too long (first effort took 24 hours, second effort with parallelization took 10 hours), we began executing on a contingency plan to get an instance in a working state with limited access to Instapaper’s archives. This short-term solution launched into production after 31 hours of downtime. The total time to create that instance and launch it into production was roughly six hours.


Our only recourse was to restore the data to an entirely new instance on a new filesystem. This was further complicated by the fact that our only interface into the hosted instances is MySQL, which made filesystem-level solutions like rsync impossible without the direct assistance from Amazon engineers.

Planet of the Apps

Husain Sumra:

Apple wasn’t the first choice for the show, according to Silverman. The project was initially shopped around to the big networks. The show drew major interest, but Will.I.Am brought up the show to Jimmy Iovine while at a meeting with Apple in Los Angeles. Apple was interested, and Silverman and the rest of the producers slowed down the process with the networks to give Apple a chance at securing it for Apple Music.


Cue later emphasized that Apple doesn’t just want to buy shows, denying that Apple was ever interested in purchasing The Grand Tour. Instead, Apple only wants to make shows that are unique and “create culture.”

The trailer is here. Most of the reactions I’ve seen have been negative:

Update (2017-02-15): See also: Nick Heer, Benjamin Mayo, Joe Rosensteel.

Update (2017-02-19): See also: Cabel Sasser.

Monday, February 13, 2017 [Tweets] [Favorites]

Swift 3 Keywords Reference

Jordan Morgan (via Ole Begemann):

So today — we’ll look at every single keyword Swift (v 3.0.1) has to offer us along with some code for each one, all in the name of booking up on our trade’s tools.

Some are obvious, some are obscure and some are sorta(ish) recognizable but they all make for great reading and learning.

Software Engineering at Google

Fergus Henderson (PDF, via Hacker News):

Write access to the repository is controlled: only the listed owners of each subtree of the repository can approve changes to that subtree. But generally any engineer can access any piece of code, can check it out and build it, can make local modifications, can test them, and can send changes for review by the code owners, and if an owner approves, can check in (commit) those changes. Culturally, engineers are encouraged to fix anything that they see is broken and know how to fix, regardless of project boundaries.


Most larger teams also have a “build cop” who is responsible for ensuring that the tests continue to pass at head, by working with the authors of the offending changes to quickly fix any problems or to roll back the offending change. (The build cop role is typically rotated among the team or among its more experienced members.) This focus on keeping the build green makes development at head practical, even for very large teams.


All code used in production is expected to have unit tests, and the code review tool will highlight if source files are added without corresponding tests. Code reviewers usually require that any change which adds new functionality should also add new tests to cover the new functionality.


Most software at Google gets rewritten every few years.

Optimizations in Syntax Highlighting

Alexandru Dima (via Hacker News):

Tokenization in VS Code (and in the Monaco Editor) runs line-by-line, from top to bottom, in a single pass. A tokenizer can store some state at the end of a tokenized line, which will be passed back when tokenizing the next line. This is a technique used by many tokenization engines, including TextMate grammars, that allows an editor to retokenize only a small subset of the lines when the user makes edits.


Holding on to that tokens array takes 648 bytes in Chrome and so storing such an object is quite costly in terms of memory (each object instance must reserve space for pointing to its prototype, to its properties list, etc). Our current machines do have a lot of RAM, but storing 648 bytes for a 15 characters line is unacceptable.


Perhaps the biggest breakthrough we've had is that we don't need to store tokens, nor their scopes, since tokens only produce effects in terms of a theme matching them or in terms of bracket matching skipping strings.


When pushing a new scope onto the scope stack, we will look up the new scope in the theme trie. We can then compute immediately the fully resolved desired foreground or font style for a scope list, based on what we inherit from the scope stack and on what the theme trie returns.

How to Stop Seeing Your Amazon Searches Everywhere

Rob Pegoraro (via Kirk McElhearn):

You can avoid this problem by doing your online shopping in a private-browsing or incognito-mode window. But it’s easy to forget to do that when you have 10 different pages open in tabs in your browser and you’re also switching between the Web, e-mail and other apps.

Instead, you can tell Amazon to stop sending you ads based on your shopping habits. To do that, visit or log into your Amazon account in a browser, click on your username in the top right corner of the page, and then click on the “Your advertising preferences” link.

You will, however, have to repeat this on each browser that you use for any Amazon shopping.

Sunday, February 12, 2017 [Tweets] [Favorites]

Virtual Apple II

Virtual Apple ][ (via Hacker News):

Almost every Apple ][ and Apple IIgs game ever made, ready to play in your browser.

I was hoping to see Life & Death. They do have have Thexder and Oregon Trail, though, along with Risk, which had a better AI than the Mac version.

Previously: Apple ][js.

Update (2017-02-12): They even have some scanned user manuals.

Testing Out Snapshots in APFS

Adam H. Leventhal:

It’s 2017, and Apple already appears to be making good on its promise with the revelation that the forthcoming iOS 10.3 will use APFS. The number of APFS tinkerers using it for their personal data has instantly gone from a few hundred to a few million. Beta users of iOS 10.3 have already made the switch apparently without incident. They have even ascribed unscientifically-significant performance improvements to APFS.


We figured out the proper use of the fs_snapshot system call and reconstructed the WWDC snapUtil. But all this time an equivalent utility has been lurking on macOS Sierra. If you look in /System/Library/Filesystems/apfs.fs/Contents/Resources/, Apple has included a number of APFS-related utilities, including apfs_snapshot (and, tantalizingly, a tool called hfs_convert).


After the volume is mounted again, not only are the contents reverted (to an empty directory in this case), but any snapshots taken after the snapshot used for the revert operation are deleted as well. One might expect APFS snapshot revert to immediately take effect and restore the contents of the volume to the previous state. Some technical issues likely make that challenging, such as what to do about programs that have files within in that volume open. So seeing if and how Apple decides to expose this functionality will be interesting.

mkfile(8) Is Severely Syscall Limited

Marcel Weiher (Hacker News):

It never occurred to me that the problem could be with mkfile(8). Of course, that’s exactly where the problem was. If you check the mkfile source code, you will see that it writes to disk in 512 byte chunks. That doesn’t actually affect the I/O path, which will coalesce those writes. However, you are spending one syscall per 512 bytes, and that turns out to be the limiting factor. Upping the buffer size increases throughput until we hit 2GB/s at a 512KB buffer size. After that throughput stays flat.


The point is that the hardware has changed so dramatically that even seemingly extremely safe and uncontroversial assumptions no longer hold. Heck, 250MB/s would be perfectly fine if we still had spinning rust, but SSDs in general and particularly the scorchingly fast ones Apple has put in these laptops just changed the equation so that something that used to just not factor into the equation at all, such as syscall performance, can now be the deciding factor.

The Slow Decline of iPad Sales

John Gruber:

But put software development aside. I think the bigger problem for the iPad is that there are few productivity tasks, period, where iPad is hardware-constrained. Aldus PageMaker shipped for the Mac in 1985. By 1987 or 1988, it was easy to argue that the Mac was, hands-down, the best platform the world had ever seen for graphic designers and visual artists. By 1991 — seven years after the original Mac — I think it was inarguable. And the improvements in Mac software during those years drove demand for improved hardware. Photoshop, Illustrator, Freehand (R.I.P.), QuarkXpress — those apps pushed the limits of Mac hardware in those days.

Michael Rockwell:

iPad owners don’t buy new iPads because the one they have is just as fast as the day they bought it. By comparison, the Windows PCs that many of these users buy are at their fastest when they’re first setup. I reference Windows users because they represent the vast majority of mainstream computer users and I believe them to be the primary reason for the massive success of the iPad in its early days.


In the tech-centric circles that many of us frequent, new hardware and software features matter, a lot. But I don’t think the mainstream user is convinced to spend hundreds of dollars on a new device just because it connects to a new kind of wireless keyboard or works with a $100 drawing accessory that you have to buy separately.


The iPad upgrade cycle might be longer than any other computing device in history. This might look terrible for Apple’s financial department, but it’s a testament to how well-crafted these devices are from both a software and hardware standpoint. The lengthy upgrade cycle lends itself to high customer satisfaction ratings and repeat customers. That’s something Apple should be proud of — a computing device that doesn’t have to be replaced every few years.

Nick Heer:

Apple has long said that the iPad’s big display provides the opportunity to create a completely different app experience. At the first Retina iPad event, Tim Cook even spent stage time mocking Android tablet apps that looked like large phone apps.

But now, five years after that event, it’s not so much the apps that are scaled-up versions of a smartphone, but rather that the operating system seems largely driven by what the iPhone can do. This was an early criticism of the iPad, but I felt it was unwarranted at the time — a larger version of a familiar interface is a great way to introduce a new product category.

Chris Adamson:

Here’s a counter-argument that is being overlooked: the iPad represents effectively all of the “productivity tablet” market[…].


Now even if the Mac sells less than the iPad, the PC market as a whole is massive… much larger than tablets, and larger still than my contrived “productivity tablet” market. And Mac’s not even 10% of this giant PC market.

So, in terms of growth opportunities, which is more realistic: finding non-tablet-users to adopt the iPad for their productivity or work needs (and making the iPad more suitable for that), or flipping more of the 90% of people already using PCs to a better version of the same thing?

Previously: Apple’s Q1 2017 Results.

Update (2017-02-12): Jeff Johnson:

iPad upgrade cycle shouldn’t be the focus. Ask why new sales aren’t growing. How did iPad reach market saturation in only 4 years?

Ole Begemann:

Not sure I buy the argument that iPad sales are slow because old devices are “fast enough”. My 3-year-old iPad Air is often painfully slow.

Friday, February 10, 2017 [Tweets] [Favorites]

Protecting Your Data at a Border Crossing

Jonathan Zdziarski:

Obviously, you want all of your devices encrypted and powered off at the border. There are plenty of ways to access content on devices (even locked ones) if the encryption is already unlocked in memory.


To lock down 2FA at a border crossing, you’ll need to disable your own capabilities to access the resources you’ll be compelled to surrender. For example, if your 2FA sends you an SMS message when you log in, either discard or mail yourself the SIM for that number, and bring a prepaid SIM with you through the border crossing; one with a different number. If you are forced to provide your password, you can do so, however you can’t produce the 2FA token required in order to log in.


I’ve written about Pair Locking extensively in the past. It’s an MDM feature that Apple provides allowing you to provision a device in such a way that it cannot be synced with iTunes. It’s intended for large business enterprises, but because forensics software uses the same interfaces that iTunes does, this also effectively breaks every mainstream forensics acquisition tool on the market as well. While a border agent may gain access to your handset’s GUI, this will prevent them from dumping all of the data – including deleted content – from it. It’s easy to justify it too as a corporate policy you have to have installed.