Friday, March 24, 2017 [Tweets] [Favorites]

APFS’s “Bag of Bytes” Filenames

David Reed (via Alastair Houghton):

I received a polite reply from my bug stating:

“iOS HFS Normalized UNICODE names, APFS now treats all files as a bag of bytes on iOS. We are requesting that Applications developers call the correct Normalization routines to make sure the file name contains the correct representation.”

This is not that surprising, technically, because HFS+ was a file system outlier, having Unicode normalization built in. Things are much easier for the file system if it can just treat names as bags of bytes.

However, Apple failed to tell developers that it was making this change when it announced AFPS at WWDC 2016. And, as far as I can tell, it’s not mentioned anywhere in the APFS guide. iOS 10.3 is scheduled to ship in a matter of weeks or months, and it will convert existing volumes to APFS. It’s not trivial to make an app that was accustomed to working with a normalized file system work without one. And since there was no announcement, I doubt most developers have even thought about this. So this is bound to cause lots of bugs.

Here are some thoughts that come to mind:

  1. Depending on what an app does, it may not care about this at all. But some apps will require major changes. Think of any code that reads a filename from within a file, or from the network, or from NSUserDefaults, or from the user’s typing, or from a URL handler, and then looks for that file by name. Or code that reads the contents of a folder and compares the filenames with ones it has seen before. Also think of any code that compares a filename with a string, or puts filenames in a dictionary or set, or creates checksums or secure hashes of filenames. Even if code is just reading a filename from disk, so that it starts out with the right byte sequence, you have to be careful of any code that processes it changing the normalization.

  2. If normalization is not handled by the file system, it becomes the responsibility of each app. Apple could update Cocoa to make the common cases easier, but it sounds like this has not been done, and it wouldn’t handle everything, anyway. Not all file system access goes through Cocoa. And, in truth, even with a file system that does Unicode normalization, some code needs to care about this.

  3. The Apple engineer’s reply is not very helpful because it’s not clear what the “correct Normalization routines” are. If APFS is not normalized, then there really is no canonical form that you can expect to find on disk. Your code has to pick one and use it consistently. Cocoa has four different methods for normalizing strings. None of these normalize the same way that HFS+ did. It uses a variant of Form D.

  4. It’s not clear what normalization the APFS to HFS+ converter uses. It doesn’t really matter, though, since apps need to be able to handle all the cases, anyway.

  5. Even if an app creates a file, I’m not sure it’s safe for it to rely on being able to find it again with the same name. The file may reside on a disk whose file system is migrated. Or the file could be restored from backup, cloud synced, or transferred in some other manner that might change its name.

  6. As far as I know, there is no API to tell, given a path, whether its file system uses normalization.

  7. As far as I know, if you have a path there is no easy way and efficient way to look for it on disk if the normalizations may differ. If you get unlucky, you would have to start at the top and read the contents of each folder until you find the file or folder that, when normalized, matches that component of your path. And there is ambiguity here because, with a bag-of-bytes file system, there could be multiple items at each level that match the path component.

  8. More generally, once APFS is deployed users can legitimately end up with multiple files in the same folder whose names only differ in normalization. So it’s not simply a matter of having all application code convert all filenames to a canonical normalization. Apps need to be able to support files like this that coexist as well as the case where there’s only one file and the name in the file system doesn’t exactly match the name that the app stored or calculated.

Update (2017-03-25): See also: Hacker News.

I should also mention that, Unicode issues aside, APFS also allows multiple files in the same folder whose names differ only in case. HFS+ does not, and depending on the variant it is not case-sensitive, either. Applications will need to handle files that sync or transfer back and forth between these systems. Some folder structures created on APFS cannot exist on HFS+. And file references that rely on case-insensitivity to work on HFS+ will not work on APFS. Cocoa does have an API to detect how the file system handles case, and apps should already have been using it, but there will probably be more people running into these issues once the iOS/Mac worlds are no longer standardized on HFS+.

Thursday, March 23, 2017 [Tweets] [Favorites]

Apple Acquires Workflow

Matthew Panzarino (tweet, MacRumors, Hacker News):

Apple has finalized a deal to acquire Workflow today — a tool that lets you hook together apps and functions within apps in strings of commands to automate tasks.

[…]

Workflow the app is being acquired, along with the team of Weinstein, Conrad Kramer, Ayaka Nonaka and Nick Frey. In a somewhat uncommon move for Apple, the app will continue to be made available on the App Store and will be made free later today.

[…]

Apple confirmed the deal, and has said the following about Workflow:

“The Workflow app was selected for an Apple Design Award in 2015 because of its outstanding use of iOS accessibility features, in particular an outstanding implementation for VoiceOver with clearly labeled items, thoughtful hints, and drag/drop announcements, making the app usable and quickly accessible to those who are blind or low-vision.”

It sounds like Apple was more keen on the accessibility than the automation.

Federico Viticci (tweet):

Workflow doesn’t need an introduction on this website. I’ve been covering the app since its debut in 2014, and have been following every update until the last major version, which brought the fantastic addition of Magic Variables. You can catch up on over two years of Workflow reporting and tutorials here, and access even more advanced workflows in our dedicated section on the Club MacStories newsletter.

At this stage, it’s not clear what Apple’s plans for Workflow in the long term might be. I have a few theories, but this isn’t the time to speculate. I’ll say this, though: Workflow has been the driving force behind my decision to embrace the iPad as my primary computer. Workflow is a shining example of the power of automation combined with user creativity and its underlying mission has always been clear: to allow anyone to improve how iOS can get things done for them in a better, faster, more flexible way. Workflow is the modern bicycle for the mind. There’s nothing else like it.

John Gruber:

This certainly provides ammunition against the argument that Apple no longer cares about power users. For me this is Apple’s most intriguing and exciting acquisition in years.

[…]

Doing things the iOS way via Workflow looks cool, but whenever it comes down to it, it always feels easier to me to just wait until I’m at a Mac and create it there.

The other way of looking at this is that perhaps it’s a sign that (a) Apple’s internal efforts are not as far along as we’d hope, and (b) that their plans for automation are more like Automator than supporting a language with inter-process communication. They may want to replace AppleScript on the Mac with something more limited, which would be unfortunate.

Stephen Hackett:

I have a million questions, many hopes and quite a few fears about this.

Michael:

The small, abused optimist in my brain hopes this will turn out as successful as Siri or TestFlight, but the pragmatist is afraid my most productive app on iOS is dead.

David Sparks:

There was another innovative application on the iPhone years ago that Apple purchased called Siri. Once they bought it, the pace of innovation slowed down and while it’s great that the Siri got incorporated into the operating system, there’s a lot of us that still miss the old version that had some crazy new innovative feature with each update. I think there’s a legitimate concern that Apple will do the same with Workflow.

[…]

If, however, Apple absorbs Workflow into the operating system with the intention of bringing real power user tools to iPhone and iPad users, I believe they could go even further than the current third-party version of Workflow. Imagine if Apple created APIs that allowed any app to tap into Workflow’s automation tools. Imagine if we could string together automation steps that allow users to press one button and have five different applications lend a hand to getting work done.

Khoi Vinh:

Unlike many Apple acquisitions, this one was not just a deal for the Workflow team but for the app itself too, which signals that Apple sees value in the product.

[…]

It now seems apparent that Apple intends to make iOS itself more open to Workflow-style automation, not less, and that’s a good thing. The kind of customization that Workflow allows is critical for the iPad, particularly, if it’s to become a true productivity platform.

[…]

When you look at competitors like Amazon’s Alexa platform, Google Assistant, Cortana, etc., they all lack a truly elegant, easy-to-master entryway for casual users who want to customize their experiences—and that’s exactly what Workflow gives Apple. (Though don’t be surprised if these other players acquire Zapier or IFTTT soon as well.)

Guy English:

This is terrific! Congratulations all! We interviewed the Workflow cats a while ago. You can listen to it here.

Gabe Weatherhead:

That doesn’t sound like a company that’s [aggressively] pursuing automation on iOS. [Accessibility] is still a wonderful, and arguably much more important, goal for Apple, but I don’t think it’s a positive move for the people that love the app. Unlike [Panzarino], I think this is bad news for anyone that relies on Workflow to make iOS useful. If all your eggs are in that one basket then you better hurry up and build a new basket. I’d love to be wrong on this front, but I don’t think automation is a priority for Apple or for iOS. The URL support isn’t just languishing on iOS, Apple has actively killed access to some [features] in launcher apps.

Nick Lockwood:

I see Apple’s acquisition of Workflow as further reinforcement of their message that 3rd party extensions of the platform are not welcome.

Most devs who try to build tools to expand the capabilities of iOS get shut down. The ones that don’t get acquired so Apple can control them.

Dr. Drang:

The big question: to incorporate it or to kill it?

Jason Snell:

I’m going to be positive and say that this is Apple acknowledging that this sort of functionality should be core to iOS, and that Workflow (or a successor) could be vastly more powerful if it’s given more power and control than any third-party app ever could. You could, in fact, argue that Workflow is the iOS app that Apple should’ve built itself, but didn’t.

Matt Birchler:

Can you imagine what Workflow could do if it had full access to the OS? I have high hopes for this move 🤞

Ole Zorn:

I hope the @WorkflowHQ acquisition leads to better automation infrastructure on iOS, and not just one privileged app, controlled by Apple.

Michael Rockwell:

I’m not worried about @WorkflowHQ going away, but I am worried about some of their actions that integrate with third-parties disappearing.

Rui Carmo:

Update: and the first post-acquisition update is already out, axing support for Pocket and Google Maps (two of the things I used with it), among others.

One can only hope that Apple has better plans for it than subsuming it and watering it down – at least they’re making it free, but I would very much like to see it evolve into something even more powerful under Apple stewardship, rather than languishing into obscurity…

Trevor Kay:

Lol didn’t take Apple long.

Ryan Burnett:

“Workflow Gallery submissions are no longer supported” 🙁

Marco Arment:

Don’t blame Workflow or Apple for removing Google, etc. actions in last night’s Workflow update.

It was lawyers, not strategy.

A few weeks ago, Workflow asked all supported apps to sign a simple 1-page contract confirming permission to call our URL schemes.

People sue Apple over ANYTHING, ALL THE TIME, so it makes sense for their lawyers to be overly cautious like this in an acquisition.

Steve Mops:

You still can provide honest release notes when you remove functionality.

Previously: Thank You, Sal.

Update (2017-03-24): See also: Accidental Tech Podcast.

Wednesday, March 22, 2017 [Tweets] [Favorites]

Kindle iOS App Adds “Send to Kindle” Feature

Tim Hardwick:

Amazon has added a long-awaited Pocket-style read-it-later feature to its Kindle iOS app that allows users browsing the web to send articles to their Kindle device for offline reading.

Amazon has offered desktop Chrome and Firefox browser extensions to feed into its Send to Kindle feature for some time, but the company had not offered a Safari-compatible iOS solution until now.

This is a good feature to have, even though I already use Instapaper, because sometimes I want to send one particular article right now. Plus, it’s easier for people who don’t already use a reading service. It also works in other apps besides Safari, and you can set it not to save the document in the cloud (so you don’t have to clear it out later).

It’s shocking that this took so long, but Amazon doesn’t seem to care that much about its iOS or Mac apps. Despite the more advanced iOS platform, the iOS Kindle app has fewer features and views than an actual Kindle. Collections-related features are particularly lacking and buggy, whereas they work perfectly from Amazon’s hardware.

Sellers Printing Counterfeit Books and Selling Under Amazon’s Brand

Bill Pollock (via Hacker News):

Images of counterfeit copies of Python for Kids being sold on Amazon. Legit copies are thicker, color, layflat binding, nicer paper.

Unfortunately, these counterfeits come directly from Amazon’s inventory. Not from a third-party seller.

Previously: Amazon Selling Fake Apple Chargers and Cables, Amazon’s Chinese Counterfeit Problem Is Getting Worse.

XCTest Closure Based Expectations

Giovanni Lodi:

XCTest provides a method to generate an expectation that will fulfil based on the result of an NSPredicate, expectation(for:, evaluatedWith:, handler:).

Add to that the fact that NSPredicate can be initialized with a closure to evaluate, and now you have a recipe for a closure based expectation.

Seriously, Google, Can You Just Make Exact Match Exact?

Daniel Gilbert:

A few years ago (2014), Google eliminated advertisers’ ability to exclude close variants as part of “exact match,” and they got away with it. People were angry. The industry suffered a blow. But people gradually moved on.

Now they’re messing with exact match again, but this time they’ve gone too far.

In the new exact match universe, an “exact match” can include close variants of the keywords and can also include the same words, but in an entirely different order. Google is denying the importance of syntax, at the expense of the industry and, ultimately, itself.

The article is about AdWords, but the general Google Web search seems to also find matches that don’t actually contain the quoted words in sequence.

Why Large Companies Acquire Small Companies

Jason Cohen:

Large acquirers don’t care about small-company financials because mathematically those won’t affect the growth or value of the acquirer.

[…]

Rather, buyer behavior is rooted in their strategy — a combination of product thesis, their theory of their market’s evolution, how they need to position for customers and against competitors, their long-term brand development, geographic expansion plans, and so on.

[…]

Pundits couldn’t agree how to analyze the sale of Instagram to Facebook. It was said that Facebook drastically overpaid (a billion dollars for a company with fanatical users but zero revenue) but also that that Instagram was stupid to sell so early (because after more “inevitable” growth it would be worth much more, and would “inevitably” be bought or go public at that larger valuation).

Monday, March 20, 2017 [Tweets] [Favorites]

Parallels Desktop Lite

Rich Trouton notes that Parallels Desktop Lite is now available in the Mac App Store. This is possible because it uses the Hypervisor framework instead of kernel extensions. So it runs in the sandbox, and you get these weird little alerts—which don’t look official, but I guess they must be—that it wants access to various parts of your system, such as “EFI”.

It’s free to run Mac and Linux VMs, and there’s a $60 In-App Purchase to run Windows.

None of the limitations compared with the Standard Edition seem to affect me. It was easy to install both macOS 10.12 and Mac OS X 10.11 by dragging and dropping the installer app—no need to create a bootable installer volume. After installing the Parallels Tools, I could copy and paste back and forth with the host OS.

It feels a bit slower than VMware to me.

Previously: VMware Fusion, Parallels Desktop 9 and Parallels Access, Turning Off Ads in Parallels.

Apple’s Next Big Thing: Augmented Reality

Mark Gurman (Hacker News):

Tim Cook has talked up a lot of technologies since becoming Apple Inc.’s chief executive in 2011. Driverless cars. Artificial intelligence. Streaming television. But no technology has fired up Cook quite like augmented reality, which overlays images, video and games on the real world. Cook has likened AR’s game-changing potential to that of the smartphone. At some point, he said last year, we will all “have AR experiences every day, almost like eating three meals a day. It will become that much a part of you.”

The technology is cool, but I just don’t see AR glasses being useful to most people most of the time.

People with knowledge of the company’s plans say Apple has embarked on an ambitious bid to bring the technology to the masses—an effort Cook and his team see as the best way for the company to dominate the next generation of gadgetry and keep people wedded to its ecosystem.

Too bad there’s no bloody ROI in an ambitious bid to fix their bug-ridden software.

But Apple really has no choice, says Gene Munster, a founding partner at Loup Ventures who covered the company for many years as an analyst. Over time, Munster says, AR devices will replace the iPhone. “It’s something they need to do to continue to grow,” he says, “and defend against the shift in how people use hardware.”

I don’t believe that.

HyperCardPreview

Pierre Lorenzi (via Stephen Hackett):

HyperCardPreview is an Mac OS X application that can display HyperCard stacks, with an look very faithful to the original. It makes the stack files alive again in the Finder with their real icons, so they don’t appear as “binaries”, and provides a QuickLook plug-in.

It can only visualize the stacks: it can’t modify them, and it can’t execute them. But it lets the user inspect the stacks, backgrounds, cards, buttons and fields, see theirs scripts and get their text contents.

It’s written in Swift, without using any deprecated APIs.

Building a Hackintosh Pro

Dan Counsell (tweet, Hacker News):

If Apple made a mini tower that was upgradable and could take a full sized graphics card (or two), I’d have purchased it in a heartbeat. However, they don’t. There’s no doubt that Apple has a refresh for the desktop market in the works, I just don’t know if it’s going to be enough to satisfy the creative market who seem to be slowly migrating to Windows.

[…]

Building a Hackintosh is not for everybody. It’s not a simple process, there is an overwhelming number of parts to choose from, and on top of this you need to pick the ones that are compatible. When you’ve built the machine you need to get macOS running on it, it’s not a quick process. If you want to do it, do your research and take your time. I’d probably say this build took me around 8 hours from unboxing the components to getting macOS installed.

I’ve been running this machine for a couple of weeks now and I couldn’t be happier. It’s super fast and I can easily switch between Mac and Windows. I’ve switched off auto-updates in Sierra. While system updates should work just fine, I prefer to hold off until the community over at tonymacx86 have confirmed there are no issues. This is probably one of the major drawbacks to running a Hackintosh.

Michael Rockwell:

The performance this machine was able to achieve, at the price he paid, is staggering. On single-core tasks, it’s faster than any Mac Apple currently sells and, if you forgo all the bells and whistles, it can be built for about $1,800.

Previously: Video Pros Moving From Mac to Windows for High-End GPUs, Building My $1,200 Hackintosh.

Update (2017-03-20): See also: Stephen Hackett, Kirk McElhearn.

Update (2017-03-22): Dave Mark:

Have your heart set on a speedy new Mac Pro? Given that nothing appears to be on the horizon from Apple, your best bet is to build your own Hackintosh.

[…]

I’d also make your way through both of the Dan case studies above just to get a sense of the process and, most importantly, to understand the caveats that come with building a Hackintosh.

Complexity and Strategy in Microsoft Office

Terry Crowley (via Simon Willison):

Unfortunately, [FrontPage’s] Preview view ended up being a gift that kept on giving. The complexity it introduced had nothing to do with any failure in the initial programming of the feature. The challenges were that as we added new functionality, Preview required special consideration — additional specification — about how it should behave and interact with these new features.

[…]

When the Word team went to estimate the cost of these features, they came back with estimates that were many times larger than PowerPoint’s estimates. Some of this cost was because the PowerPoint code architecture was better structured to add these types of visual features. But the bulk of the growth in estimates was because Word’s feature set interacted in ways that made the specification (and hence the implementation) more complex and more costly.

[…]

What I found is that advocates for these new technologies tended to confuse the productivity benefits of working on a small code base (small N essential complexity due to fewer feature interactions and small N cost for features that scale with size of codebase) with the benefits of the new technology itself — efforts using a new technology inherently start small so the benefits get conflated.

[…]

The framework typically fails to evolve along the path required by the product — which leads to the general advice “you ship it, you own it”. This means that you eventually pay for all that code that lifted your initial productivity. So “free code” tends to be “free as in puppy” rather than “free as in beer”.

[…]

In fact, it was clear that [Google Docs was] using an asymmetric technical attack of leveraging their simplicity to deliver sharing and co-editing features. These features were clearly differentiated and would be immensely hard to deliver on top of the existing highly functional (large N) Office apps. As we started building the web apps, we needed to decide whether we were going to “walk away” from our own complexity like we had when we developed OneNote or embrace the existing complexity, with the costs and constraints that that would imply for future development.

Friday, March 17, 2017 [Tweets] [Favorites]

Rotate to Pixel Zoom

Keir Thomas:

Now repeat all the above, but this time rotate the image back to the way it was (that is, you’ll need to tap the rotate button three times). Tap Done.

Main feature illustration You will now be able to zoom into the image via the pinch-expand gesture, and will be able to keep going all the way down to pixel detail – although the pixels will be blurred because of the way iOS handles zoom.

Google Home Now Testing Audio Ads

Katyanna Quach (via John Gruber):

Promos for Disney’s new Beauty and the Beast flick, released in American cinemas today, are being mixed into Google Home’s responses to questions. In a test with a Home owned by a Reg writer, the chatbot device started touting the kids movie in between telling the time and news headlines. This is with the default configuration: no opt-ins or opt-outs.

[…]

Google’s official response to this is absolutely pants-on-head mad:

This isn’t an ad; the beauty in the Assistant is that it invites our partners to be our guest and share their tales.

Nick Heer:

I don’t care how much storytelling tinsel an advertisement happens to be wrapped in — an ad is an ad.

I looked through Google’s support documentation for Home and even downloaded the app looking for anything that would specifically state that the device could be used for advertising. Nothing on the Google Home website implies that ads would start running on the device.

Update (2017-03-20): Nick Heer:

Disney may not have paid Google to tell Home users about their new movie, but that’s what it felt like to a lot of people.

Netflix to Replace Star Ratings With Thumbs Up/Down

Tim Hardwick (via Luc Vandal, Hacker News):

Previous star ratings given by users will be used to personalize their Netflix profiles, but the ability to rate a TV series or movie by awarding stars is set to disappear altogether, according to Variety.

[…]

According to Netflix, at one point subscribers had awarded over 10 billion 5-star ratings and more than half of all members had rated more than 50 titles. However, the company eventually concluded that star ratings had become less relevant, with some users giving documentaries 5 stars and silly movies just 3 stars, even though they would watch the silly movies more often than the highly rated documentaries.

Janko Roettgers:

“We are addicted to the methodology of A/B testing,” Yellin said. The result was that thumbs got 200% more ratings than the traditional star-rating feature.

I don’t think people agree on what a rating is supposed to mean, anyway. Secondly, the thumbs aren’t really a binary system because you can choose not to rate at all. So it seems to me that they’re effectively just getting rid of the 2- and 4-star options.

Update (2017-03-19): Jeff Johnson:

I think movies should be rated by how many times you’d watch them. 0 if you regret watching, 1 for a decent movie, and going up to infinity.

Update (2017-03-20): John Gruber:

For a personally curated collection, 5-star ratings can be meaningful. But for a recommendation service that averages ratings among all users, they are not. It’s the difference between designing for the ideal case of how people should behave versus designing for the practical case of how people actually behave.

Kirk McElhearn:

Binary ratings make a lot more sense in certain contexts, and with YouTube, it’s a natural fit. You don’t rate a movie on YouTube; you generally rate a cat video, a TED Talk, or something short.

I disagree that this type of rating will work on Netflix.

Making an External Display a Monitor, Not a TV

Erica Sadun:

What I didn’t expect was how awful the text looked on it. I hooked up the monitor to the MBP using my Apple TV HDMI cable. The text was unreadable. I use similar TV-style monitors for my main system and they display text just fine. However, I’m using normal display ports and cables for my mini. This is the first time I’ve gone HDMI direct.

[…]

All the searches lead to this ruby script. The script builds a display override file containing a vendor and product ID with 4:4:4 RGB color support. The trick lies in getting macOS to install, read, and use it properly. That’s because you can’t install the file directly to /System/Library/Displays/Contents/Resources/Overrides/ in modern macOS. Instead, you have to disable “rootless”.

System Level Breakpoints in Swift

Daniel Jalkut (tweet):

So, as a rule, Swift programmers who want to be advanced debuggers on iOS or Mac platforms, also need to develop an ability for mapping Swift method names back to their Objective-C equivalents. For a method like UIView.layoutSubviews, it’s a pretty direct mapping back to “-[UIView layoutSubviews]”, but for many methods it’s nowhere near as simple.

[…]

If the API has been rewritten using one of these rules, it’s almost certain that the Swift name of the function is a subset of the ObjC method name. You can probably leverage the regex matching ability of lldb to zero in on the method you want to set a breakpoint on[…]

Thursday, March 16, 2017 [Tweets] [Favorites]

Foundation Swift Archival & Serialization Proposal

Itai Ferber et al. (via Swift Evolution):

Foundation’s current archival and serialization APIs (NSCoding, NSJSONSerialization, NSPropertyListSerialization, etc.), while fitting for the dynamism of Objective-C, do not always map optimally into Swift. This document lays out the design of an updated API that improves the developer experience of performing archival and serialization in Swift.

Specifically:

  • It aims to provide a solution for the archival of Swift struct and enum types
  • It aims to provide a more type-safe solution for serializing to external formats, such as JSON and plist

Dell Precision 5520

Daniel Rubino (via Steve Lubitz):

The Precision 5520 was supplied by Dell for this review. It features a 4K IGZO display, 32GB of RAM, Intel Xeon processor, NVIDIA Quadro graphics and 512GB of Class 50 solid-state drive (SSD).

The starting price for the Precision 5520 is $1,399. The unit we evaluated in this review retails for $2,867.

It also has room for a second hard drive or SSD, real function keys, a T arrow key arrangement, and a trackpad that clicks. It is about the same weight as the 2012 Retina MacBook Pro and gets the same battery life (7 hours).

The most similar Mac is a $3,099 MacBook Pro with a lower resolution display, 16 GB of RAM (max), a 2.9 GHz i7, and a Radeon Pro 460.

Previously: The Curious State of Apple Product Pricing, New MacBook Pros and the State of the Mac.

Swift Compiler Not Optimized for Lots of Cores

Jacek Suliga (via Peter Steinberger):

When LinkedIn embraced Swift and rebuilt its flagship iOS app using (mostly) Swift, I spent a good amount of time profiling the new compiler characteristics. At that time (mid 2015), it was clear that the compiler was CPU-bound (I/O and memory had no visible impact on build speed). However, the build times were determined primarily by the number of cores and threads available to the compiler, more so than by the CPU clock speed.

Based on this analysis, we decided to use 12-core MacPros for core development, and we saw a significant (2-3 times) speedup of our builds compared with quad-core laptops.

But now:

As you can see, 12-core MacPro is indeed the slowest machine to build our code with Swift, and going from the default 24 jobs setting down to only 5 threads improves compilation time by 23%. Due to this, even a 2-core MacMini ($1,399.00) builds faster than the 12-cores MacPro ($6,999.00).

[…]

For machines with more than 4 cores consider reducing number of concurrent jobs allowed for Xcode - until the concurrency issue described above has been resolved.

Marc Knaup:

Be careful with disabling Spotlight indexing for ~/Library/Developer as it will also exclude dSYM symbol files of builds and archives.

Xcode uses Spotlight to search for matching dSYM files when symbolicating crash reports. Turning off Spotlight indexing for them might break crash symbolication completely.

Update (2017-03-16): Joe Groff:

This smells like a hardware or kernel issue.

Blaming any particular component is premature. “There Is A Bug Exposed By Swift 3 Running on Sierra” is all we know.

Core count doesn’t seem to matter either, we observed the same issue on 4-core Xeon setups. Seems to be Xeon-specific.

The bug report is here.

See also: Colin Cornaby.

Guetzli: A New Open Source JPEG Encoder

Robert Obryk and Jyrki Alakuijala:

Guetzli [guɛtsli] — cookie in Swiss German — is a JPEG encoder for digital images and web graphics that can enable faster online experiences by producing smaller JPEG files while still maintaining compatibility with existing browsers, image processing applications and the JPEG standard. From the practical viewpoint this is very similar to our Zopfli algorithm, which produces smaller PNG and gzip files without needing to introduce a new format; and different than the techniques used in RNN-based image compression, RAISR, and WebP, which all need client and ecosystem changes for compression gains at internet scale.

[…]

Guetzli specifically targets the quantization stage in which the more visual quality loss is introduced, the smaller the resulting file. Guetzli strikes a balance between minimal loss and file size by employing a search algorithm that tries to overcome the difference between the psychovisual modeling of JPEG's format, and Guetzli’s psychovisual model, which approximates color perception and visual masking in a more thorough and detailed way than what is achievable by simpler color transforms and the discrete cosine transform.

Previously: Brotli, Zopfli, JPEG Image Compression, Lepton Image Compression.

Update (2017-03-22): John Gruber:

There is, of course, a catch. Image processing is really slow. It takes about 8 minutes for Guetzli to process a single iPhone camera image on my 5K iMac.

Wednesday, March 15, 2017 [Tweets] [Favorites]

The Curious State of Apple Product Pricing

Neil Cybart:

Given the preceding list, a strong case could have been made for Apple to price its new wireless headphones at $249, or even $299. The fact that Samsung priced its Gear IconX at $199 seemed to suggest a sub-$200 retail price for AirPods was unlikely. Instead, Apple sent shockwaves pulsing through the market by pricing AirPods at only $159. The action instantly removed all available oxygen from the wireless headphone space. The idea of Apple coming out with a new product that would underprice nearly every other competitor was unimaginable ten years ago.

[…]

At $269, Apple Watch Series 1 is one of lowest-priced smartwatches worth buying in the marketplace. Attractive pricing was one key factor driving record Apple Watch sales this past holiday quarter. In fact, even the Apple Watch Series 2, at $349, is one of the lowest-priced smartwatches in its class[…]

However, it seems like most Mac models are not as competitively priced right now, Apple is focusing on the high end of the tablet market, and it sounds like iPhone prices will be expanding up rather than down.

Update (2017-03-17): See also: MacRumors.

John Gruber:

AirPods are still showing a delivery estimate of “6 weeks”. Either demand remains unexpectedly strong or production remains unexpectedly difficult (or some combination of both).

That One Optional Property

Soroush Khanlou:

Other code in the view controller switched on the message’s existence to determine which views to allocate, how to lay them out, and so on. The optionality of the property represents more than existence of a simple string. It had implications in the rest of the view controller and the rest of the view layer.

The more essential point here is that the string now no longer represents a string that might be there or might not — it now represents the presentation style or mode that the view controller is in. Was it created in the context of a push notification or through normal browsing? The answer is in this barely related property!

His solution is to replace it with an enum or a promise, both of which have other benefits besides clarity.

Allo Can Reveal What You’ve Searched For

Tess Townsend:

A unique feature of Allo is that you can use Assistant while in the middle of a conversation with a friend. You could, for example, ask Assistant to search for restaurants in a certain area, while you’re talking to a friend about where to eat.

But Assistant isn’t perfect, and sometimes it responds with answers unrelated to questions at hand, or it will respond with an answer to an earlier question — and it’s then that it can inadvertently reveal a previous search query.

[…]

Google responded to our story: “We were notified about the Assistant in group chats not working as intended. We’ve fixed the issue and appreciate the report.”

Previously: Google Reneges on Allo Privacy Feature.

Looking at Model-View-Controller in Cocoa

Matt Gallagher:

When Cocoa says Model-View-Controller it’s mostly trying to evoke the notion of Separated Presentation and Content in application design (the idea that the model and view should have a decoupled design and be loosely linked at construction). To be fair, it’s not just Cocoa that uses Model-View-Controller in this way: most modern uses of the term are really intended to convey Separated Presentation rather than the original Smalltalk-80 definition.

[…]

The important point to note is that the controller is the center of the object graph with most communication passing via the controller – distinct from the Smalltalk-80 version where the model was the center of the graph.

[…]

In my opinion, the critical failing for Cocoa Bindings remains the difficulty in adding custom transformations and custom properties. These could both be done but the work involved in registering transformers and exposing bindings dictionaries made a tiresome affair. It always just seemed easier to pass the data through the view controller without bindings. This meant that Bindings tended to help the simplest problems (that didn’t need much help) but didn’t have much impact on the harder problems.

Since Cocoa Bindings in Mac OS X 10.3, there haven’t really been any clear attempts by Apple to alter the design pattern used for Cocoa apps.

Marcel Weiher:

The problem with using the same name is shadowing: since the names are the same, accessing the original definition is now hard. Again, this wouldn’t really be a problem if it weren’t for the fact that the old MVC solved exactly the kinds of problems that plague the new MVC.

Previously: Model Widget Controller (MWC) a.k.a.: Apple “MVC” Is Not MVC.

The Story of Firefox OS

Ben Francis (Hacker News):

During this five year journey hundreds of members of the wider Mozilla community came together with a shared vision to disrupt the app ecosystem with the power of the open web. I’d like to reflect on our successes, our failures and the lessons we can learn from the experience of taking an open source browser based mobile operating system to market.

[…]

In a rush to get to market we imitated the app store model with packaged apps, adding to the problem we set out to solve. Lost focus on what makes the web the web (URLs). Too much focus on the client side rather than the server side of the web stack.

[…]

After realising “open” on its own doesn’t sell, ultimately chose a strategy to compete mainly on price, which is just a race to the bottom. Suffered from a lack of product leadership and direction after the initial launch.

Tuesday, March 14, 2017 [Tweets] [Favorites]

RxNot

Chris Adamson:

It’s not like iOS isn’t meant to deal with asynchronous events. In fact, it has lots of different techniques… all mutually incompatible, owing to the long development of the Cocoa and Cocoa Touch APIs over the years, and the legacy of Objective-C.

[…]

So now let’s look at how ReactiveX deals with this. In Rx, things that change over time are offered as observables. […] So, where’s the rub? For starters, it takes a long time to get the hang of Rx, both to get into its mindset (it sucks when you just need to get the current value of something, but it’s an observable, so there’s no real concept of a “current” value). And then there are all the operators. Follow that link to the operators that I linked a few paragraphs back. There are over 70 of them.

[…]

In practice, there are lots of ways to mess this up. Are you right to only want one event? What if you forget to capture self weakly? The worse problem is when your closure is never called: setting breakpoints in this code will do nothing, so it’s a hair-pulling exercise to figure out if fooObservable is not producing events, or if your chain is screwing them up somehow.

[…]

In my experience, Rx turns out to be far more costly than it would originally appear from the propaganda. I can’t speak for my ex-employer of course, but my own takeaway is that 1) adopting Rx can take way longer than you’d expect, 2) RxSwift seems to really slow down the Swift compiler (possibly because of having to do a bunch of type inference through those chains of Rx operators).

Mark Lilback:

I don’t buy the idea of replacing everything with it. But use the right tool for the right job. For chained operations it is better than promises/futures and much better than callbacks/closures.

How to Set Up and Use Airline Boarding Passes in Wallet

Adam C. Engst:

I’m sure many of you are nodding your heads in agreement, or wondering why I’m telling you all this. I’m no frequent flyer, so I’m sure that these improvements have been obvious to road warriors for a while. But I wanted to document how to do all this because I saw a lot of people still relying on paper boarding passes on that last trip. The experience of relying on an iPhone is so good, and so much better than dealing with paper, that if you’ve avoided it so far, I urge you to give it a try next time. But feel free to print a paper boarding pass as a backup until you’re comfortable with using your iPhone instead!

[…]

At the end of the check-in process, the apps I’ve used have provided a relatively clear Add to Wallet link or button. Tap it, and when it hands you off to Wallet, make sure everything looks correct in the boarding pass and then tap the Add link in the upper-right corner. You may or may not need to do this for each boarding pass; check to see what’s in Wallet after the first one. Now you’re ready for when you go to the airport.

I’d still want a paper copy in case my iPhone spontaneously shuts down. Also, sometimes they won’t let you swipe the pass yourself, and I don’t want to hand my iPhone to the attendant. I admit to not really appreciating the benefit of Wallet here. It’s an extra step to use, but then you can access the boarding pass from the lock screen instead of having to open the app. And maybe it saves you from having to rely on the airline app at the critical moment? I’ve twice had an airline app stop working after arriving at the airport but before boarding. So now I always take a screenshot of the boarding pass just in case.

Eject from Interface Builder

Eject (via Brian King):

One common pain point with Interface Builder is that as a view becomes more dynamic and is managed more programatically, Interface Builder becomes less helpful. This tool lets developers use Interface Builder without that concern, giving them an Eject button to hit when Interace Builder starts getting in the way, and provides an easy path to transition to full programatic view layout.

Previously: Working Without a Nib.

Dropbox Drops Public Folders

Josh Centers:

On 15 March 2017, Dropbox will convert all Public folders on free accounts to private folders, breaking existing links. Dropbox now recommends using a shared folder or shared link for sharing files with others. Dropbox Plus and Dropbox Business users can continue to use the Public folder until 1 September 2017.

[…]

Although it’s annoying to lose the Public folder and its HTML rendering capability, Dropbox remains useful because it’s integrated so well into the Mac and iOS experience, and it just works.

Although, some of its recent behavior has been troubling. I just want a basic folder that syncs and doesn’t peg my CPU when there’s filesystem activity in an unrelated folder.

Previously: Dropbox Discontinues HTML Rendering.

Stop Auto-Play Web Videos

Kirk McElhearn:

Auto-play videos suck. They use bandwidth, and their annoying sounds get in the way when you’re listening to music and open a web page. I happen to write for a website that uses them, and it annoys me to no end. (My editors have no control over those auto-play videos, alas.)

But you can stop auto-play videos from playing on a Mac. If you use Chrome or Firefox, it’s pretty simple, and the plugins below work both on macOS and Windows; if you use Safari, it’s a bit more complex, but it’s not that hard.

To do the whole thing from Terminal:

defaults write com.apple.safari WebKitMediaPlaybackAllowsInline 0
defaults write com.apple.safari com.apple.Safari.ContentPageGroupIdentifier.WebKit2AllowsInlineMediaPlayback 0

(Or substitute com.apple.SafariTechnologyPreview.)

Jason Snell:

I fought for six months to get those [videos] turned off. When I left, they turned ’em right back on.

Monday, March 13, 2017 [Tweets] [Favorites]

Sketch’s Open File Format

Jasim A Basheer (via Andy Matuschak):

“enables more powerful integrations for third-party developers” is stating it lightly. This is what the fine folks at Bohemian Coding has done — they opened up Sketch’s file format into a neat JSON making it possible for anyone to create and modify Sketch compatible files.

[…]

Can you imagine what kind of new things will now be possible? One word: design automation (okay, two words!).You want Artboards that showcase a font and its variations, like a Google Fonts page? There’s probably going to be a script to generate that file. There will be websites from which you can download freshly brewed Sketch files based on what you ask — say an image gallery, or a landing page, or a signup form. You’ll be able to pick your brand colors, choose a theme, randomize it, and voila! you have a Sketch design to start playing with. Someone could even build a Sketch equivalent that runs on the browser. The possibilities are many!

[…]

And Sketch itself comes with an in-built REPL where anyone can whip up a plugin in no time, and just hit Save to package it into a distributable plugin. There is nothing extra to do — it is all in the app. This makes it very easy for users of Sketch to get started with plugin development, and they happily take the bait!

[…]

But I wanted to make a point with all this: even though plugin development is undocumented and painful, developers still build crazy useful things on top of it.

What Twitter Means to Me

Ricky Mondello:

When I was in high school in a small town in upstate New York, I didn’t really have anyone around to help develop or even share my interest in technology with. Twitter was my connection to the world I wanted to live in. Although I’d been a member of several forums in the past, I liked Twitter more than any forum because there was no pretense of being limited to any particular topic. In 2008, Twitter was accessible on my iPod touch in a way that other communities weren’t. From that iPod, I followed people who talked about Mac software, making web pages, podcasting, and politics, and that stream of information helped me figure out what I wanted to do with my future.

[…]

What started as a way for me to fill a void in the types of people I knew in “real life” changed as I left that small town. Today, Twitter is how I get my news.

Previously: Anil Dash’s Advice for Twitter.

Update (2017-03-20): See also: Stephen Hackett.

Sierra Logging Spew

Lloyd Chambers:

Opening Console, you’ll see the ceaseless spew from a cornucopia of processes, including many I never want, and will never use. It might be ‘quiet’ at times, but what I’ve found is that a number of Apple services get triggered from time to time to go into a state of endless bitching and moaning, often with messages that equate to “fix this bug someday”.

For example, here is this lovely new Apple bug involving touchui. On a Mac Pro no touch user interface exists, but the engineers at Apple don’t bother to test much any more, so the com.apple.nowplayingtouchui apparently is just going to fail forever.

The spew has been getting worse the last few versions of macOS, and it’s also more of a problem with 10.12 and its new logging subsystem. I’m running into more cases of runaway logging where the same message is repeated multiple times per second. Not only does this hog the CPU, but Console falls behind so that (even with a filter to hide the spew) it can take minutes for actual, relevant log entries to show up. The only remedy seems to be to restart the Mac.

A Look Inside Backblaze

Backblaze (via Hacker News):

Files can be written to the Vault even if a Storage Pod is down with two parity shards to protect the data. Even in the extreme — and unlikely — case where three Storage Pods in a Vault are offline, the files in the vault are still available because they can be reconstructed from the 17 available pieces.

[…]

We use Reed-Solomon erasure encoding. It’s a proven technique used in Linux RAID systems, by Microsoft in its Azure cloud storage, and by Facebook too. The Backblaze Vault Architecture is capable of delivering 99.99999% annual durability thanks in part to our Reed-Solomon erasure coding implementation.

[…]

We developed our Reed-Solomon implementation as a Java library. Why? When we first started this project, we assumed that we would need to write it in C to make it run as fast as we needed. It turns out that modern Java virtual machines working on our servers are great, and just-in-time compilers produces code that runs pretty quick.

Yev:

Yes we do plan on expanding to more datacenters, and we do have emergency plans in place, though we do choose our datacenters carefully to make sure that we avoid any natural-disaster prone areas. As for backing up our own data - we certainly do make backups of our core info/necessary data. As for the user data that we store, that’s backed up across the storage pods in a vault as discussed in that post. We do not replicate customer data across multiple datacenters. At our price-point, that’s just not feasible.

Encrypted Media Extensions (EME)

Glyn Moody (Reddit):

For the last four years, the Web has had to live with a festering wound: the threat of DRM being added to the HTML 5 standard in the form of Encrypted Media Extensions (EME). Here on Techdirt, we’ve written numerous posts explaining why this is a really stupid idea, as have many, many other people. Despite the clear evidence that EME will be harmful to just about everyone -- except the copyright companies, of course -- the inventor of the Web, and director of the W3C (World Wide Web Consortium), Sir Tim Berners-Lee, has just given his blessing to the idea[…]

Tim Berners-Lee:

The question which has been debated around the net is whether W3C should endorse the Encrypted Media Extensions (EME) standard which allows a web page to include encrypted content, by connecting an existing underlying Digital Rights Management (DRM) system in the underlying platform. Some people have protested “no”, but in fact I decided the actual logical answer is “yes”.

[…]

The reason for recommending EME is that by doing so, we lead the industry who developed it in the first place to form a simple, easy to use way of putting encrypted content online, so that there will be interoperability between browsers.

[…]

If W3C did not recommend EME then the browser vendors would just make it outside W3C. If EME did not exist, vendors could just create new Javascript based versions. And without using the web at all, it is so easy to invite ones viewers to switching to view the content on a proprietary app. […] If the Director Of The Consortium made a Decree that there would be No More DRM in fact nothing would change. Because W3C does not have any power to forbid anything.

Anne van Kesteren:

The fact that the CDM (DRM code in the article) is not part of the standard means the promise of interoperability is false.

And the fact that CDM sandboxing is not defined means you allow for a race to the bottom in terms of end-user security.

Peter Bright:

EME does not specify any DRM scheme per se. Rather, it defines a set of APIs that allow JavaScript and HTML to interact with decryption/protection modules. These modules will tend to be platform-specific in one way or another and will contain the core DRM technology.

[…]

The EFF, along with the Free Software Foundation (FSF) and various other groups, has campaigned against the development of the EME specification. They signed an open letter voicing their opposition and encouraged others to sign a petition against the spec.

[…]

However, it’s not clear that EME does anything to exacerbate that situation. The users of EME—companies like Netflix—are today, right now, already streaming DRM-protected media. It’s difficult to imagine that any content distributors that are currently distributing unprotected media are going to start using DRM merely because there’s a W3C-approved framework for doing so.

Facebook Bots Relearn Early PC Lessons

Sarah Perez (via Farshad Nayeri):

Facebook’s Messenger bots may not be having the impact the social network desired. Just yesterday, online retailer Everlane, one of the launch partners for the bot platform, announced it was ditching Messenger for customer notifications and returning to email. Following this, Facebook today announced an upgraded Messenger Platform, which introduces a new way for users to interact with bots: via simple persistent menus, including those without the option to chat with the bot at all.

One of the problems with Facebook’s bots is that it’s often unclear how to get started. The directory of bots in Messenger wasn’t initially available and now only reveals itself when you start a search in the app. And it hasn’t always been obvious how to get a bot talking, once added, or how to navigate back and forth through a bot’s many sections.

Previously: Bots Won’t Replace Apps.

Friday, March 10, 2017 [Tweets] [Favorites]

Making Swift Enums and Structs Equatable

Ole Begemann:

The downside of not having a default case is, of course, more boilerplate to write. […] This isnʼt fun to write, and it would get even worse with more cases. The number of states the switch statement must distinguish grows quadratically with the number of cases in the enum.

[…]

You can make this considerably more manageable with some intelligent application of the _ placeholder pattern. While we saw above that a single default clause is not enough, one pattern per case is.

Ole Begemann:

Unfortunately, like the enum example I talked about in the previous post, this conformance to Equatable is very fragile: every time you add a property to the struct, you have to remember to also update the implementation of the == function. If you forget, your Equatable conformance will be broken, and depending on how good your tests are this bug has the potential to go undetected for a long time — the compiler wonʼt be able to help you here.

[…]

It occurred to me to use the standard libraryʼs dump function as a safeguard. dump is interesting because it uses Swiftʼs reflection capabilities to create a string representation of a value or object that includes all storage fields.

[…]

The biggest drawback of the solution might be that dump is not a perfectly reliable way to determine equality. It should be pretty good at avoiding false negatives, but youʼll probably see some false positives, i.e. values that really are equal but whose dump outputs are different.

Rust’s Language Ergonomics Initiative

Aaron Turon (via Joe Groff, Hacker News):

There are three dimensions of the reasoning footprint for implicitness:

  • Applicability. Where are you allowed to elide implied information? Is there any heads-up that this might be happening?
  • Power. What influence does the elided information have? Can it radically change program behavior or its types?
  • Context-dependence. How much of do you have to know about the rest of the code to know what is being implied, i.e. how elided details will be filled in? Is there always a clear place to look?

The basic thesis of this post is that implicit features should balance these three dimensions. If a feature is large in one of the dimensions, it’s best to strongly limit it in the other two.

The ? operator in Rust is a good example of this kind of tradeoff. It explicitly (but concisely) marks a point where you will bail out of the current context on an error, possibly doing an implicit conversion on the way. The fact that it’s marked means the feature has strongly limited applicability: you’ll never be surprised that it’s coming into play. On the other hand, it’s fairly powerful, and somewhat context-dependent, since the conversion can depend on the type where ? is used, and the type expected in the scope it’s jumping to. Altogether, this careful balance makes error handling in Rust feels as ergonomic as working with exceptions while avoiding some of their well-known downsides.

St. Clair Software Leaves the Mac App Store

Jon Gotow:

For both applications, complying with Apple’s sandboxing and feature constraints to get them approved for sale would have required significant rewrites. And in Jettison’s case, it would also require that buyers download a separate helper app to enable its full functionality. I realize that some people will be put off or inconvenienced by the fact that these apps are no longer in the Mac App Store – my apologies if you’re one of those folks, but it just doesn’t make sense for Jettison and HistoryHound.

Their other two products are App Tamer, which I’ve been meaning to try to reduce Dropbox’s CPU use, and the venerable Default Folder X. Those are probably impossible to sandbox.

Update (2017-03-11): See also: Sip and Videoloupe (via Bad Uncle Leo).