Archive for July 2015

Friday, July 31, 2015

NSTaggedPointerString

Mike Ash (comments):

Thus we can see that the structure of the tagged pointer strings is:

  1. If the length is between 0 and 7, store the string as raw eight-bit characters.
  2. If the length is 8 or 9, store the string in a six-bit encoding, using the alphabet "eilotrm.apdnsIc ufkMShjTRxgC4013bDNvwyUL2O856P-B79AFKEWV_zGJ/HYX".
  3. If the length is 10 or 11, store the string in a five-bit encoding, using the alphabet "eilotrm.apdnsIc ufkMShjTRxgC4013"

[…]

The five-bit alphabet is extremely limited, and doesn’t include the letter b! That letter must not be common enough to warrant a place in the 32 hallowed characters of the five-bit alphabet.

[…]

Because this table is used for both six-bit and five-bit encodings, it makes sense that it wouldn’t be entirely in alphabetical order. Characters that are used most frequently should be in the first half, while characters that are used less frequently should go in the second half. This ensures that the maximum number of longer strings can use the five-bit encoding.

Address Sanitizer

WWDC 2015 Session 413 (video, PDF):

Address Sanitizer is an LLVM tool for C-based languages, that serves the same purpose as Guard Malloc, as it finds memory errors at runtime, however, it has a lot of benefits over the other tools.

It has much less runtime overhead and it also produces comprehensive, detailed diagnostics that are integrated directly into the Xcode UI.

What’s also very important is that it is the only such tool that works on iOS devices.

Nigel Timothy Barber:

The Xcode 7 Address Sanitizer is a revelation. Essential tool.

Mike Ash:

Address Sanitizer uses a simple but brilliant approach. It reserves a fixed section within the process’s address space called the shadow memory. In Address Sanitizer terms, a byte that is marked as forbidden is “poisoned,” and the shadow memory tracks which bytes are poisoned. A simple formula translates each address within the process’s address space into a spot in the shadow memory. Each eight-byte chunk of regular memory maps to a byte of shadow memory, which tracks the poison state of those eight bytes.

[…]

With this table structure in place, Address Sanitizer generates extra code in the program to check every read and write through a pointer, and throw an error if the memory in question is poisoned. This is the advantage of being integrated into the compiler and not merely existing as an external library or runtime environment: every pointer access can be reliably identified and the appropriate checks added into the machine code.

Compiler integration also allows neat tricks like the ability to poison and guard local and global variables, not just heap allocations. Locals and globals are allocated with a bit of extra padding in between them, and the padding is poisoned to catch any overflows. This is something that Guard Malloc can’t do, and that Valgrind has difficulty with.

Keith Harrison:

It is important to understand that the Address Sanitizer is a run-time tool so you need to exercise your code for it to find problems. This makes it a good candidate to enable when running unit and UI tests. Whilst Apple claims it has low overhead (2x-5x CPU and 2x-3x memory) it is not performance free so you may want to experiment if you have large test suites before leaving it enabled.

Objective-C Improvements and Swift Interoperability

WWDC 2015 Session 401 (video, PDF):

Go to any Objective-C header and choose the ‘show related items’ button in the top-left corner. This will bring down a menu of related items, one of which is ‘generated interface.’ And this will show you the Swift mapping for that header. This is the exact same view that you got in Xcode 6 using the ‘jump to definition’ feature, but now you can get it easily from any header in your target.

[…]

Now, on both of these examples, I have shown that these methods in Objective-C have multiple parameters, only one of which is the error parameter, but there are also cases where methods only have one parameter and you will get something like ‘check resource is researchable and return error.’ As you can see in Swift, since we already know that the method can return an error from that ‘throws’ keyword, we will chop off those last three words for you, just for you!

I guess that answers my question.

Jordan Morgan:

The first step to making this better is one of my favorite new Objective-C features, nullability annotations.

[…]

Though it took about thirty-two years, generics is now a thing in Objective-C.

[…]

Essentially, we’ve informed the compiler that the property and its collection will have some kind of a UIView. This allows for more information in the type contract that we’ve not had before.

The real treat? Implicit downcasting.

drekka:

There’s a couple of this going on here. Firstly you will notice that we are not using ObjectType anywhere. It appears that the name of a Generic reference is only usable in the header of a class. I’ve tried all sorts of syntaxes to see if there was a way to declare and use it in the implementation, but I’ve been unsuccessful. So I’ve concluded that Objective-C Generics simply don’t make the placeholders available to implementations. Unfortunately I cannot confirm against Apples code.

What I have concluded is that when a Generic is resolved by the compiler, it resolves to the top most class that can replace it. Normally this is id, however bounded Generics can make this more specific. See Restricting Generics below.

Joe Groff:

ObjC generics also can’t be part of the canonical type system without breaking C++ templates

Bitcode

drfuchs:

I managed to ask Chris Lattner this very question at WWDC (during a moment when he wasn’t surrounded by adoring crowds). “So, you’re signaling a new CPU architecture?” But, “No; think more along the lines of ‘adding a new multiply instruction’. By the time you’re in Bitcode, you’re already fairly architecture-specific” says he.

Wolf Rentzsch:

surprised most dev gripes about bitcode is unpredictable optimization effects. Folks, we’ve been living in emulated ISAs since the 90s

I think bitcode is a huge win. About only thing I’ll miss is cool ISA-specific instructions. Now reliant on OS vendor providing access.

Landon Fuller:

I’m a lot less worried about emulated ISAs given that the chances I have to debug one are pretty much nil.

Bitcode: non-reproducible Apple-internal toolchain bugs, emergent bugs from undefined behavior that previously worked, etc ...

dshirley:

When it becomes a requirement to submit apps in bitcode format, how will this impact architecture specific code (ie. assembly, or anything that is ifdef’d for that matter). It makes sense that assembly isn’t converted to bitcode, but doesn’t everything need to be in bitcode in order for an archive to be fully encoded in bitcode? I have an app that’s hitting a compile warning when archiving complaining that a specific 3rd party library doesn’t contain bitcode so the app cannot be archived with bitcode. That 3rd party library won’t emit bitcode ostensibly because it contains assembly (I could be wong about the cause, though).

Rainer Brockerhoff:

I suppose this would also allow Swift ABIs to change at any time, without dylibs in the app.

See also Accidental Tech Podcast episodes 122, 123, and 124.

Update (2015-09-25): Alex Denisov:

This picture clearly demonstrates how communication between frontend and backend is done using IR, LLVM has it’s own format, that can be encoded using LLVM bitstream file format - Bitcode.

Just to recall it explicitly - Bitcode is a bitstream representation of LLVM IR.

Frederic Jacobs:

Bitcode will enable support for better microarchitecture support but gets nowhere close to target independence. Applications compiled for the armv7 target could still run on armv7s devices but additional optimisations make applications faster if they contain a armv7s slice. The advantage that Bitcode provides on top of app thinning is negligible in my opinion since it will only provide a slight speed up until the developer uploads a new build with the optimized slice.

[…]

The centralization of the building and signing process is what worries me: an adversary could find a vulnerability in the LLVM backend to obtain remote code execution on Apple’s Bitcode compilation infrastructure to inject a compiler trojan that would affect every single app on the App Store that was submitted with Bitcode.

Falsehoods Programmers Believe

A bunch of links via Jeff Atwood:

Update (2018-07-26): Dave DeLong:

Your calendrical fallacy is thinking…

Update (2019-05-16): Alex Chan (via Hacker News):

These three facts all seem eminently sensible and reasonable, right?

  1. Unix time is the number of seconds since 1 January 1970 00:00:00 UTC
  2. If I wait exactly one second, Unix time advances by exactly one second
  3. Unix time can never go backwards

False, false, false.

Apple Music Matches Files With Metadata Only

Kirk McElhearn (comments):

If you’ve used iTunes Match in the past, you may know that it matches music using acoustic fingerprinting, which means that iTunes scans the music, and matches it to the same music. It doesn’t matter what tags files have: you could have, say, a Grateful Dead song labeled as a song by 50 Cent, and iTunes Match will match the Grateful Dead song correctly. (Here’s how Wikipedia defines acoustic fingerprinting.)

Apple Music, however, works differently. It does not use the more onerous (in time and processing power) acoustic fingerprinting technique, but simply uses the tags your files contain. And it can lead to errors. Here’s an example of how this can be a bit surprising.

Update (2015-07-31): Marco Arment:

This is embarrassing. No wonder people have had so many problems and so much data loss with Apple Music’s cloud-library features.

It’s as if nobody who made this implementation decision had ever encountered remasters, re-recordings, clean versions, live performances, or the many other extremely common reasons why two very different audio recordings might have the same artist and title.

[…]

Don’t let these cloud-matching “features” anywhere near your music collection.

Update (2015-08-01): Kirk McElhearn:

I’ve been unable to reproduce this issue, and my guess is that there was a glitch with Apple’s servers that has since been corrected. If you only subscribe to Apple Music, or are using it on a free trial, then your songs are matched using metadata only. If you subscribe to both iTunes Match and Apple Music, then iTunes matches your songs using digital fingerprinting.

Marco Arment:

That this happened at all (and I got reports from many other people who were affected) means that iTunes Match is less trustworthy as primary storage, and it never really was trustworthy as primary storage because it has always been buggy and inconsistent, so my recommendations remain to avoid letting these features integrate with your music collection.

Serenity Caldwell:

Apple’s been doing Match (as have other companies) for four years now and none of them have it anywhere near perfect.

Marco Arment:

I wonder if matching is still the right choice. I bet most people’s music libraries are smaller than their photo libraries.

Update (2015-08-03): Kirk McElhearn:

FYI, I was able to reproduce the change-the-metadata-and-match thing in iTunes this morning. I made a screen recording of it.

SQLite FTS5

SQLite Release Notes:

Added the experimental FTS5 extension. Note that this extension is experimental and subject to change in incompatible ways.

SQLite FTS5 Extension:

The principle difference between FTS3/4 and FTS5 is that in FTS3/4, each instance-list is stored as a single large database record, whereas in FTS5 large instance-lists are divided between multiple database records. This has the following implications for dealing with large databases that contain large lists:

  • FTS5 is able to load instance-lists into memory incrementally in order to reduce memory usage and peak allocation size. FTS3/4 very often loads entire instance-lists into memory.

  • When processing queries that feature more than one token, FTS5 is sometimes able to determine that the query can be answered by inspecting a subset of a large instance-list. FTS3/4 almost always has to traverse entire instance-lists.

  • If an instance-list grows so large that it exceeds the SQLITE_MAX_LENGTH limit, FTS3/4 is unable to handle it. FTS5 does not have this problem.

[…]

FTS5 has no matchinfo() or offsets() function, and the snippet() function is not as fully-featured as in FTS3/4. However, since FTS5 does provide an API allowing applications to create custom auxiliary functions, any required functionality may be implemented within the application code.

The set of built-in auxiliary functions provided by FTS5 may be improved upon in the future.

Thursday, July 30, 2015

Disabling Versions and Reclaiming Space

I have been having problems with the system Versions feature. On one volume, nearly half the total space is consumed by the versions database. I think it’s supposed to automatically prune old versions if more space is needed, but it doesn’t, so I instead get errors from MoneyWell that it can’t save its database.

The versions database is managed by revisiond, but as far as I can tell there are no built-in tools for controlling it. The only way to manually reclaim space seems to be to delete the .DocumentRevisions-V100 folder, which will delete all the versions.

I would like to selectively disable the Versions feature for certain volumes, but there doesn’t seem to be a way to do that. As far as I can tell, there isn’t even an official switch to disable it entirely. This answer suggests checking “Ask to keep changes when closing documents,” but that doesn’t work. This answer says that you can set the ApplePersistence default in Terminal. At least that works on a per-app basis, because it can also make some apps like TextEdit not function properly.

But the bottom line is that I wouldn’t have a problem with Versions if it would automatically compact its database.

Update (2015-07-30): Here is John Siracusa’s discussion of the versions (a.k.a. document revisions) feature.

Dixie

Skyscanner:

Dixie is an open source Objective-C testing framework for altering object behaviours. Test your app through creating chaos in the inner systems. The primary goal of Dixie is to provide a set of tools, which the developers can test their code with. Behind the goal is the ideology of “do not always expect the best”. You can read more about this here.

Via iOS Dev Weekly:

I found that the best way to think about Dixie was to see it as like a mocking framework that you enable globally inside your app rather than only in your test target. This allows various parts of your app to be replaced for integration tests, amongst other things.

How GameplayKit AI Works

Paul Hudson:

Amongst the many features introduced in GameplayKit, one of the most immediately useful is its ability to provide artificial intelligence that can evaluate a situation and make smart choices. We’re going to be using it in our Four in a Row game to provide a meaningful opponent, but first it’s essential that you understand how GameplayKit tackles the AI problem because it directly affects the code we’ll write.

[…]

GameplayKit’s artificial intelligence works through brute force: it tries every possible move, then tries every possible follow-on move, then every possible follow-on follow-on move, etc. This runs up combinations extremely quickly, particularly when you consider that there are 4,531,985,219,092 unique positions for all the pieces on the board! So, you will inevitably limit the depth of the search to provide just enough intelligence to be interesting.

UIKonf 2015 Videos

Apple’s Plan for More Connected Apps

Federico Viticci:

At WWDC 2015, Apple announced app search, a new feature of iOS 9 that will help users find content inside apps. Beyond the user-facing aspects of a new search page on iOS and proactive suggestions from Siri, however, lies a commitment to fundamentally rethink iOS’ relationship with apps and the web, with deep implications for the future.

[…]

With local app search, iOS 9 can build an index of content, app features, and activities that users may want to get back to with a search query. Built like a database and already in use by Apple apps such as Mail and Reminders, CoreSpotlight will provide low level access to the index of an iOS device, making it easy to organize and retrieve content users have previously seen, created, or curated.

[…]

The way Apple has designed local app search is reminiscent of viewing history in web browsers: users want to be able to find things they’ve seen in the past; from this standpoint, local app search will be more akin to a local app history, enabling users to find apps, but also files, pages, profiles, likes, comments, and other types of previously seen content.

[…]

Apple is building a cloud index of user activities and web content. As the company describes it, the cloud index will “inform any searches by any user of your app on any device”, and it’s broadly aimed at creating a public and anonymous index of what users look for and engage with. Part web crawling and part a crowdsourced search database, Apple’s cloud index will be closely tied to apps and results from it will be displayed in the search page of iOS 9.

[…]

Developers have choice when it comes to the cloud index. In an iOS app, they can tag user activities as public, and public activities get sent to Apple's cloud index as anonymous hashes.

It’s interesting that CoreSpotlight does not seem to require creating lots of stub files like Spotlight on the Mac.

Wednesday, July 29, 2015

App Store Invoice JavaScript Injection

Darren Pauli (via Ole Begemann):

“After we received no serious reply, we released the data,” Mejri told El Reg in an email. Apple did not respond to a request for comment, and it’s not clear if the vulnerability has been addressed.

In a nuthsell, the bug works like this: you change the name of your iThing to include JavaScript code, then download or purchase an app from either the Mac or iTunes stores. Apple's systems generate an invoice, and email it to you and make a copy available online from your store account.

That JavaScript code stashed in your device name will be embedded in the invoice, so opening it in a browser will execute it, allowing it to attempt to do bad things like hijack your Apple account. Sellers and Apple staff viewing a copy of the invoice will also get attacked.

Google Cloud Storage Nearline Graduates to General Availability

Avtandil Garakanidze (via @arqbackup):

We’re introducing on-demand I/O to allow you to increase I/O in situations where you need to retrieve data from a Google Cloud Storage Nearline bucket faster than the provisioned 4 MB/s read throughput per TB of data stored. On-demand I/O delivers improved throughput when you need it and you only get charged for what you use. You now can access your data in seconds, while relying on predictable data recovery.

NSValue and Boxed Expressions

Alex Denisov (via iOS Dev Weekly):

Few hours ago I finally finished with my patch to Clang. It took a lot of time, but for me it is the most interesting and challenging OSS contribution so far.

I’m not going to dive deep into the details, but will give an overview of the new feature it brings to Objective-C.

Another approach is to use macros. Scott Morrison uses the ternary operator to swallow the @ so that you can write code like @point(1,2).

The Force Touch Trackpad

Fraser Speirs:

This thing is seriously, seriously brilliant and, in my opinion, a subtle but significant step forward in the Mac experience.

[…]

My big insight into the Force Touch trackpad is that you never need to use your thumb for clicking - ever. The reason we used the thumb was because the click was only effective in the lower quarter of the trackpad due to the hinge. As long as your tracking speed is set high enough that you can go from one side of the screen to the other in one movement, you can just use your pointing finger.

If you think about it, this really takes the Mac another step closer to more direct manipulation of objects on screen.

Coping With iTunes Features That Have Disappeared

Kirk McElhearn:

Whenever a new version of iTunes rears its head, new features are added, but also some features get cut. In this week’s column, I look at several of these features that are missing in action. In some cases—such as shuffling all music by an artist in the iOS Music app, or streaming Beats 1 radio over AirPlay—I offer a workaround. For another, I can only lament that certain useful features have been axed.

Phasing Out Google+

Bradley Horowitz:

People have told us that accessing all of their Google stuff with one account makes life a whole lot easier. But we’ve also heard that it doesn’t make sense for your Google+ profile to be your identity in all the other Google products you use.

So in the coming months, a Google Account will be all you’ll need to share content, communicate with contacts, create a YouTube channel and more, all across Google. […] As always, your underlying Google Account won’t be searchable or followable, unlike public Google+ profiles.

Update (2015-08-14): Seth Fiegerman (via Todd Ditchendorf):

“Vic was just this constant bug in Larry’s ear: ‘Facebook is going to kill us. Facebook is going to kill us,’” says a former Google executive. “I am pretty sure Vic managed to frighten Larry into action. And voila: Google+ was born.”

[…]

The slow demise of Google+ sheds light on how a large technology company tries and often fails to innovate when it feels threatened. The Google+ project did lead to inventive new services and created a more cohesive user identity that continues to benefit Google, but the social network itself never truly beat back existing rivals. Facebook is now larger than ever, with 1.4 billion users and a market capitalization more than half of Google’s. It continues to poach Google employees. Facebook and Twitter are also slowly chipping away at Google’s dominance in display ad revenue.

Tuesday, July 28, 2015

Why JSON Doesn’t Have Comments

Douglas Crockford (via Dave Dribin):

First, comments turned out to not be very useful.

Second, comments were the most difficult feature to support in JSON parsers. By removing comments, support for JSON in other languages became much easier.

Third, removing comments aligns JSON more closely with YAML, allowing YAML to become a JSON superset. Generally, standards tend to splinter. It is interesting to see these two merging.

Fourth, I saw examples of comments used to carry information, with an expectation that applications would have to have custom comment parsers and generators. That was not a pattern that I wanted to encourage.

Douglas Crockford (via Dave Dribin):

I removed comments from JSON because I saw people were using them to hold parsing directives, a practice which would have destroyed interoperability. I know that the lack of comments makes some people sad, but it shouldn’t.

Update (2015-08-01): Hacker News comments.

Flickr Pro Is Back

Flickr:

Today, we’re excited to unveil the new Flickr Pro for both existing and new members of our community, a subscription that offers improved stats, brings back our Pro badge, offers a discount on Adobe software and more.

[…]

Flickr members who don’t currently have Flickr Pro can purchase it today for $5.99/month or $49.99/year.

First, there was regular (free) and Pro ($25/year for no ads, unlimited storage, and extra features). Then there was Free, Ad Free ($50/year for no ads), and Doublr ($499/year for 2 TB instead of 1 TB). Now, Ad Free is called Pro again and adds the statistics features, free shipping, and a 20% discount on the Adobe Creative Cloud plan for Photoshop and Lightroom. The other difference is that the old Ad Free account only removed ads when you were logged in, whereas with the new Pro account:

Your photos, whether viewed by you or a fan, will never be shown next to an advertisement.

Which is a good thing because the ads seem to have gotten a lot more intrusive lately. And I often view my photostream when logged out to make sure that the privacy settings are correct.

Intel Compute Stick

David Pogue:

It’s called the Intel Compute Stick, and it’s really small (4 x 1.5 x 0.5 inches) — so small, you might mistake it for a USB flash drive. It’s the first Windows PC you might lose in your bag. If you were a movie character being chased by techno-terrorists, you could slip the whole thing into your sock.

Better yet: It’s not only the smallest Windows PC type in the world; it’s also the least expensive: just $150 (with 2GB of memory, 32GB of storage, and Windows 8.1).

[…]

That’s right. Instead of plugging a monitor into your PC, you now plug your PC into the monitor.

Checking If Mission Control Is Active

Tim Schröder:

Unfortunately, Apple has never provided a public API to programmatically check if Mission Control is active.

[…]

It turns out that there is nevertheless an easy way to check if Mission Control is active (code snippet below): We just ask which windows are present on the screen. If there is a specific window present owned by the dock process and with a vertical origin of -1.0, this will be a “Dock Window” as shown by Mission Control only […] Of course this is slightly hackish and might break in the future […]

Swift Whole Module Optimization

Keith Harrison:

For the None or Fast optimization levels the compiler is working with a single source file at a time. When the definition of the generic function is in the same source file as the calling code the compiler knows we are calling the function with integers and optimizes away unnecessary object handling code. This optimization known as Generic Specialization will only work when the compiler has visibility of the function definition meaning that the external definition runs more slowly.

It makes compilation slower, though.

Monday, July 27, 2015

Safari 9’s Responsive Web Design Mode

What’s New in Safari (via Daniel Jalkut):

The Responsive Design Mode provides a fast-switching layout preview of your webpage across a spectrum of viewport sizes. This viewport quick-look tool helps you quickly spot layout issues on a webpage across many screen sizes. It includes several preset sizes for Apple devices, shown in Figure 3, and allows custom viewport sizes to be defined.

Cool.

Apple at Its Most Pompous

Apple:

Also amazing? The fact that there are over a million and a half capable, beautiful, inspiring apps on the App Store. And each and every one was reviewed and approved by a team of real live humans. With great taste. And great suggestions. And great ideas.

John Gruber:

What irks here, fundamentally, is that Apple is taking credit for the great apps in the App Store, rather than giving credit to the third-party developers who make them.

Nick Heer:

Apple has gotten really, really good at making ads that show the products, rather than talking about them — remember the “Every Day” series? They’re still doing ads like that series, but for the Watch. And, while I know they have to change things up every so often, these feel like a miss to me. They’re too smug, and too pompous for my tastes. I prefer the softer sell.

Ember Leaves the Mac App Store

Nik Fletcher:

One of the biggest problems we want to fix is sync. Formal sync in Ember right now, given the typical size of an Ember library, hasn’t performed as well as it should have - and we know that you also want to use other services besides iCloud and Dropbox. So, instead of just the two sync options we’ll be giving you the option to both relocate the library, as well as have multiple Ember libraries! For those of you wanting to use Google Drive or some other service you can - just place your Ember library in the chosen location.

[…]

It will be a paid upgrade, and we’ll be making sure that upgrade pricing (for Mac App Store and direct customers) is a no-brainer once the public beta concludes.

We will only offer Ember 2 directly, so every Ember customer can receive upgrade pricing. No MAS-ing about ;)

Previously: Ember and iCloud Core Data.

Linked Lists, Enums, Value Types and Identity

Airspeed Velocity (tweet):

So, with our nodes being references, we essentially got a free globally-unique identifier for each one, which we could then use to implement node equality. But now that we have switched away from box, we can’t use === any more. So what can we do?

[…]

Finally, a third option: tag our list elements with a number, and use that to compare them.

I hope there’s a better way.

Friday, July 24, 2015

Security and Privacy Changes in iOS 9

Alban Diquet:

To further prevent user tracking, Apple has extended MAC address randomization to additional services, and seems to now include location services scans.

[…]

Keychain items can now be encrypted using both the device’s passcode and an “Application password”; both values are then needed to decrypt and retrieve the item. This allows Apps to control when the data is accessible/decrypted, instead of having the data decrypted as soon as the device is unlocked.

[…]

iOS 9 brings several new extension points geared toward network filtering and VPNs:

  • The Packet Tunnel Provider extension point, to implement the client-side of a custom VPN tunneling protocol.
  • The App Proxy Provider extension point, to implement the client-side of a custom transparent network proxy protocol.
  • The Filter Data Provider and the Filter Control Provider extension points, to implement dynamic, on-device network content filtering.

These extension points can only be used with a special entitlement, thereby requiring Apple to approve the extension first.

Maybe someday Little Snitch for iOS will be possible.

Weak and Unowned References in Swift

Hector Matos:

At this point we have a retain cycle. You see, closures in Swift behave exactly like blocks in Objective-C. If any variable is declared outside of the closure’s scope, referencing that variable inside the closure’s scope creates another strong reference to that object. The only exceptions to this are variables that use value semantics such as Ints, Strings, Arrays, and Dictionaries in Swift.

Here, NSNotificationCenter retains a closure that captures self strongly when you call eatHuman(). Best practice says that you clear out notification observers in the deinit function. The problem here is that we don’t clear out the block until deinit, but deinit won’t ever be called by ARC because the closure has a strong reference to the Kraken instance!

Other gotchas where this could happen is in places like NSTimers and NSThread.

The fix is to use a weak reference to self in the closure’s capture list. This breaks the strong reference cycle.

[…]

Weak and unowned references are essentially the same. It does not increase the retain count of the object being referred. However, in Swift, an unowned reference has the added benefit of not being an Optional.

Code Signing in El Capitan

Craig Hockenberry:

Gatekeeper rejected the app because I’m using Sparkle 1.5b6. The framework has symlinks to paths on a Mac that I doubt Andy Matuschak uses anymore. That’s completely reasonable: those symlinks could just as easily point to something a lot more damaging that a non-existent directory.

The --strict option currently checks the validity of symlinks. You can point a symlink at a path in your own application package, /System or /Library, but nowhere else. The code signing rules also allow language translations to be removed, but if they are present they must be unmodified.

[…]

Many of your customers will be downloading and running your app on the El Capitan public beta: you should do the codesign --deep --strict check on any new releases to avoid customer support issues. It’s also likely that a similar check will be performed for you when the Mac App Store eventually allows submissions for 10.11.

Don’t Use GUIDs As Passwords

Raymond Chen:

This is a really bad idea. GUIDs are designed for uniqueness, not for security.

For example, we saw that substrings of GUIDs are not unique. For example, in the classic v1 algorithm, the first part of the GUID is a timestamp. Timestamps are a great technique for helping to build uniqueness, but they are horrifically insecure because, well, duh, the current time is hardly a secret!

DYLD_PRINT_TO_FILE Local Privilege Escalation Vulnerability

Stefan Esser (via Rene Ritchie, comments):

When this variable was added the usual safeguards that are required when adding support for new environment variables to the dynamic linker have not been used. Therefore it is possible to use this new feature even with SUID root binaries. This is dangerous, because it allows to open or create arbitrary files owned by the root user anywhere in the file system. Furthermore the opened log file is never closed and therefore its file descriptor is leaked into processes spawned by SUID binaries. This means child processes of SUID root processes can write to arbitrary files owned by the root user anywhere in the filesystem. This allows for easy privilege escalation in OS X 10.10.x.

At the moment it is unclear if Apple knows about this security problem or not, because while it is already fixed in the first betas of OS X 10.11, it is left unpatched in the current release of OS X 10.10.4 or in the current beta of OS X 10.10.5.

[…]

The problem with this code is that it does not come with any safeguards that are required when adding new environment variables to the dynamic linker. Normally for security reasons the dynamic linker should reject all environvent variables passed to it in case of restricted files. This is automatically handled when new environment variables are added to the processDyldEnvironmentVariable() function. However in the DYLD_PRINT_TO_FILE case the code was directly added to the _main function of dyld.

Update (2015-08-13): This is fixed in the final release of Mac OS X 10.10.5.

Update (2015-08-21): Dan Goodin (via Roustem Karimov):

On Monday, researchers from anti-malware firm Malwarebytes said a new malicious installer is exploiting the vulnerability to surreptitiously infect Macs with several types of adware including VSearch, a variant of the Genieo package, and the MacKeeper junkware.

Strings in Swift 2

Apple (tweet):

In Swift 2, the String type no longer conforms to the CollectionType protocol, where String was previously a collection of Character values, similar to an array. Now, String provides a characters property that exposes a character collection view.

[…]

When you add an element to a collection, you expect that the collection will contain that element. That is, when you append a value to an array, the array then contains that value. The same applies to a dictionary or a set. However, when you append a combining mark character to a string, the contents of the string itself change.

[…]

However, String determines equality based on being canonically equivalent. Characters are canonically equivalent if they have the same linguistic meaning and appearance, even if they are composed from different Unicode scalars behind the scenes. […] In Swift, strings are considered equal regardless of whether they are constructed from decomposed or precomposed character sequences.

Update (2015-07-25): Pierre Lebeaupin:

If Apple can’t get its “Characters”, UTF-16 scalars, and bytes of a seemingly simple string such as “café” straight in a blog post designed to show these very views of that string, what hope could you possibly have of getting “character”-wise text processing right?

Indeed, Apple has corrected its blog post (silently).

marcelolr:

Baking in a canonicalization policy into a basic type is … interesting. I’m sure somewhere someone will be unhappy their password no longer hashes to the same value.

Thursday, July 23, 2015

Dynamic Swift

Lee Morgan:

We’ll go ahead and extend our KVC protocol and add our default implementation.

extension KVC {
    
    func valueForKey(key : String) -> Any? {
    
        let mirror = reflect(self)
        
        for index in 0 ..< mirror.count {
            let (childKey, childMirror) = mirror[index]
            if childKey == key {
                return childMirror.value
            }
        }
        return nil
    }
}

David Owens II has a different implementation that also handles setting values:

As you might be gathering, the basic premise is to use a backing store to maintain our values and type information. The implementation can then verify that the data coming in is correct.

David Owens II:

But wait… if we can shadow the functions and replace them with our own implementations, then this means that method swizzling is back on the table!

There are several caveats, though.

Adrian Kashivskyy:

Remember that reflection in Swift is currently read-only and there’s no way to modify your program at runtime (with Objective-C-derived classes being an exception as you can still class_addMethod() in Swift).

Brent Simmons asks how to instantiate an arbitrary class at runtime.

David Owens II:

The same way as C++, an ugly switch statement!

Wil Shipley:

On the latest Swift you have to add the word “init” to make this more explicit. Then it works. e.g:

let a:P = clsA.init(i:1)

Brent Simmons:

Jim takes this one step further: what if all you have is the class name as a string?

Alas, that’s currently only possible using Objective-C classes. There is no NSClassFromString() for Swift. [Update (2015-07-23): Actually, there is. See the comments below.]

David Owens II:

Xcode 7 Beta 4 is out and it is a doozy! One of the changes is that performSelector is now available from Swift. Now, this isn’t going to make your Swift types dynamic all of a sudden. However, what it does open the door for, is writing both your ObjC-style code and your Swift code all in the same language: Swift.

Brent Simmons:

Here’s the deal I’m willing to make: to make my code more elegant (read: smaller, easier to maintain, reusable), I’m willing to let it crash if I’ve configured things incorrectly.

[…]

So my case is that Swift itself needs things like this. If not a clone of KVC, then something very much like it.

Rob Rhyne:

When Apple announced the Swift programming language in 2014, there were questions about the future of frameworks that depended heavily on the dynamic nature of Objective-C. Frameworks such as Core Data marshall objects with type dependencies determined at runtime. The strict type checks required by the Swift compiler appeared in conflict with Core Data.

[…]

It turns out the answer to one question begat the solution to the other. By turning this class extension into a protocol and with a clever use of typealias, I resolved the insidious casting problem while maintaining type safety. Protocols to the rescue!

In other words, there are ways to use Core Data from Swift in a type-safe way. But it would still be impossible to write Core Data itself in pure Swift.

init? vs. init throws

David Owens II:

What I’ve come to realize is, my problem is really about the callsite, I actually always want to use throws. The simple fact is that I can return more information about the failure and let the caller decide if they care about it or not.

But if the caller doesn’t care about it, this makes the callsite ugly.

Ok, so let’s make it better:

func trythrow<T>(@autoclosure fn: () throws -> T?) -> T? {
    do { return try fn() }
    catch { return nil }
}

func example3() {
    guard let tt = trythrow(try Foo(throwable: 0)) else {
        print("error")
        return
    }
    
    print("Foo(throwable:) creation passed")
    
}

This gets us into a happy place again. We can turn failures where we do not care about the specific reasons into a code path that is more natural. If this becomes a pattern, it begs the question why this wouldn’t be just built into the language.

This thread suggests doing that by adding try?, which looks like a good idea.

Perl 6 Due This Year

Larry Wall (via Hacker News, Slashdot):

So natural language is very flexible this way because you have a very intelligent listener – or at least, compared with a computer – who you can rely on to figure out what you must have meant, in case of ambiguity. Of course, in a computer language you have to manage the ambiguity much more closely.

Arguably in Perl 1 through to 5 we didn’t manage it quite adequately enough. Sometimes the computer was confused when it really shouldn’t be. With Perl 6, we discovered some ways to make the computer more sure about what the user is talking about, even if the user is confused about whether something is really a string or a number. The computer knows the exact type of it. We figured out ways of having stronger typing internally but still have the allomorphic “you can use this as that” idea.

[…]

Part of the reason the Perl 6 has taken so long is that we have around 50 different principles we try to stick to, and in language design you’re end up juggling everything and saying “what’s really the most important principle here”? There has been a lot of discussion about a lot of different things. Sometimes we commit to a decision, work with it for a while, and then realise it wasn’t quite the right decision.

We didn’t design or specify pretty much anything about concurrent programming until someone came along who was smart enough about it and knew what the different trade-offs were, and that’s Jonathan Worthington. He has blended together ideas from other languages like Go and C#, with concurrent primitives that compose well. Composability is important in the rest of the language.

Keyboard Maestro 7

Stairways Software:

Trigger your macros with hot keys that are down, pressed, released, or tapped multiple times. Or trigger with a change in window focus or folder or clipboard.

Each action now includes a Gear menu to enable, disable, try, rename, add notes, set colors, or get help, as well as copying the action as text or an image for easier sharing.

[…]

As you type text tokens, functions, variables and search tokens possible completions appear right at the cursor. Select the one you need and have it entered for you.

[…]

The new Mouse Display window shows the mouse coordinates as your cursor moves around the screen. Set the corner of screen or window to measure from, then copy the results into your action.

I fell out of the habit of using this type of utility more than a decade ago. Lately, I’ve been thinking that I’ve been missing out.

The Lagging Mac App Store

Craig Hockenberry:

Mac developers have never had access to TestFlight, either internally or externally. It’s “coming soon”, and until that day comes, there’s no way to test apps that use the iCloud servers. Which sucks for both the developer and the customer.

[…]

[Analytics] is nowhere to be found on the Mac App Store. Again, it’s “coming soon”.

Just yesterday, Apple did something great for developers. They now block reviews on beta OS releases. Unless that operating system is for the Mac.

Update (2015-07-24): Daniel Jalkut:

But there is no Twitter account for the Mac App Store, and the @AppStore account has nearly never mentioned Mac software. (Go ahead, do an advanced Twitter search for “from:AppStore mac“). Around four years ago at WWDC, I asked a group of App Store affiliated engineers what they made of this. They all looked at me blankly, paused, looked at each other blankly, paused, and then shrugged as if to say “Huh, I wonder why we don’t promote the Mac App Store?” I don’t know either! But they were all in a much better position than I to find out or push for a change. Evidently, that hasn’t happened.

[…]

I guess it’s lucky the Mac was around for so many years before the dawn of iOS, it’s had time to accumulate many developer-empowering features. Although Apple’s priorities with respect to development resources and marketing seem to be focused on iOS today, we enjoy many privileges on the Mac that I doubt iOS developers will ever see.

[…]

I think this speaks to the likely truth that Apple is, more than anything, under-staffed and not well situated to deploy solutions to both platforms in tandem.

Update (2015-07-28): Wil Shipley:

See also: bundle pricing, symbolicating crashes, TestFlight, subscriptions, gifting apps…

Web Design: The First 100 Years

Maciej Ceglowski:

Today I hope to persuade you that the same thing that happened to aviation is happening with the Internet. Here we are, fifty years into the computer revolution, at what feels like our moment of greatest progress. The outlines of the future are clear, and oh boy is it futuristic.

But we’re running into physical and economic barriers that aren’t worth crossing.

We’re starting to see that putting everything online has real and troubling social costs.

And the devices we use are becoming ‘good enough’, to the point where we can focus on making them cheaper, more efficient, and accessible to everyone.

So despite appearances, despite the feeling that things are accelerating and changing faster than ever, I want to make the shocking prediction that the Internet of 2060 is going to look recognizably the same as the Internet today.

Unless we screw it up.

And I want to convince you that this is the best possible news for you as designers, and for us as people.

Wednesday, July 22, 2015

Apple Music: Problems Adding Albums, Songs Deleted

Jim Dalrymple:

My problems started about a week after installing Apple Music. While Apple Music Radio and Playlists worked well, adding music to my library is nothing short of a mind-blowing exercise in frustration.

I started to notice that whenever I added an album to my library, not all of the songs would get added. When I looked at the list of songs, there would be some missing—sometimes, most of the album would be missing.

[…]

I know I’m not the only one having this problem. There are threads on Apple’s support forums detailing similar issues to the ones I’m having, and I’ve noticed tweets in my stream reporting the same problems.

At some point, enough is enough. That time has come for me—Apple Music is just too much of a hassle to be bothered with. Nobody I’ve spoken at Apple or outside the company has any idea how to fix it, so the chances of a positive outcome seem slim to none.

As if all of that wasn’t enough, Apple Music gave me one more kick in the head. Over the weekend, I turned off Apple Music and it took large chunks of my purchased music with it. Sadly, many of the songs were added from CDs years ago that I no longer have access to. Looking at my old iTunes Match library, before Apple Music, I’m missing about 4,700 songs. At this point, I just don’t care anymore, I just want Apple Music off my devices.

Kirk McElhearn:

I’ve had many of the same problems, and I understand exactly how Jim feels. For some of us, our music libraries are sacred, and Apple has damaged collections that we’ve been maintaining for a long time.

Update (2015-07-23): Jason Snell:

What’s funny is, when I saw that Jim was ripping into Apple Music, I thought it would be for a completely different reason that he didn’t even mention!

Yesterday, Apple had a pretty severe cloud-services outage. And with it went my access to Apple Music.

Nick Heer:

For what it’s worth, this sounds like an iCloud Music Library problem, not an Apple Music issue. It’s splitting hairs, but it’s an important distinction to make. Because I have Apple Music turned on, but not iCloud Music Library, I get all of the streaming features, but none of the saving or syncing ones. That means my local files remain untouched, which gives me a vastly greater sense of security.

Update (2015-07-27): Jim Dalrymple:

I arrived at Apple this morning to talk to them about my issues with Apple Music and to hopefully fix my problems. The good news is that I have about 99 percent of my music back.

Let’s get the easy stuff out of the way. The missing and duplicate song issues that we’ve all seen in Apple Music are being fixed shortly. They are certainly aware of what’s been going on, I can assure you.

[…]

Apple said my music was never deleted and that it was in the cloud the entire time. Before Apple Music, iTunes Match would show me all of my songs—matched, uploaded, and purchased. However, if you turn off iCloud Music Library and Apple Music, iTunes Match will only show your purchased content now. There is no way to separate iTunes Match from the iCloud Music Library. Before, you would turn off iTunes Match—now you would turn off iCloud Music Library.

[…]

However, I’m still missing a couple of hundred songs. Apple’s theory is that I deleted them—that when I was trying to fix Apple Music, I mistakenly deleted my own files. While I concede that it is within the realm of possibility that I deleted my own files, it doesn’t make sense to me.

Kirk McElhearn:

What Jim says about iTunes Match and iCloud Music Library isn’t exactly correct (Apple Music does not include iTunes Match, but it matches tracks, albeit in a different way). Here’s an article I wrote for Macworld that explains the (admittedly confusing) situation.

StruckPaper:

And if one isn’t Jim Dalrymple, what would have happened? SOL?

Marco Arment:

With the introduction of Apple Music, Apple confusingly introduced a confusing service backed by the iTunes Store that’s confusingly integrated into iTunes and the iOS Music app (don’t even get me started on that) and partially, maybe, mostly replaces the also very confusing and historically unreliable iTunes Match.

So iTunes is a toxic hellstew of technical cruft and a toxic hellstew of UI design, in the middle of a transition between two partly redundant cloud services, both of which are confusing and vague to most people about which songs of theirs are in the cloud, which are safe to delete, and which ones they actually have.

Even Jim’s follow-up piece, after meeting privately with Apple in PR-damage-control mode, is confusing at best about what actually might have happened, which is completely understandable because it sounds like even Apple isn’t sure.

John Gruber:

As clear as mud how this all works. Why not make Apple Music a separate standalone app? Apple Music: subscription service with DRM. iTunes: music you own, no DRM.

iOS 9 to Block Abuses of -canOpenURL:

Juli Clover:

Currently, companies like Twitter and Facebook are able to see which apps you have downloaded on your iPhone or iPad, sometimes using that information to deliver targeted ads. In The Information’s example, Facebook and Twitter might use your app data to see that you have a lot of games downloaded, offering up an ad for a game you don't yet have installed.

Kurt Wagner (via Nick Heer):

Twitter announced on Wednesday that its advertisers can use that app information to target users with ads. Marketers will be able to see the different categories of apps you have downloaded onto your phone as well as how recently you downloaded them in order to understand what you’re interested in.

Greg Pierce:

The key bits regarding changes to URL schemes in iOS 9 is the Privacy and Your Apps session, starting at around the 9 minute mark under the heading “App Detection”.

There are two URL-related methods available to apps on iOS that are effected: canOpenURL and openURL. These are not new methods and the methods themselves are not changing. As you might expect from the names, “canOpenURL” returns a yes or no answer after checking if there is any apps installed on the device that know how to handle a given URL. “openURL” is used to actually launch the URL, which will typically leave the app and open the URL in another app.

Up until iOS 9, apps have been able to call these methods on any arbitrary URLs. Starting on iOS 9, apps will have to declare what URL schemes they would like to be able to check for and open in the configuration files of the app as it is submitted to Apple.

[…]

I have independent confirmation from several sources that these limitations are meant to only apply to “canOpenURL” and it is a bug that they are also effecting “openURL”. There are still implications, and many apps will need some updates, but that’s not so dramatic a change.

[…]

Even if that is the case, loss of the ability to call “canOpenURL” on arbitrary URLs will limit the user experience on many automation applications of URLs because an app has no way of testing if the URL can be opened successfully and will have to rely on the system to report errors.

Update (2015-07-22): Federico Viticci:

iOS 9 beta 4 has a new alert to confirm launching URL schemes for the first time. Choice is remembered later.

Update (2015-09-08): Keith Harrison:

With the schemes included in Info.plist everything works as before. When you link against iOS 9 you are not limited to 50 distinct schemes you just need to declare what you need in Info.plist. There seems to be no limit for how many schemes you can include but I would expect questions from the App Store review team if they think you are abusing the mechanism.

Flashlight Brightens Up Spotlight in Yosemite

Josh Centers:

Developer Nate Parrot has created Flashlight, an open platform that makes it possible for developers to expand Spotlight’s repertoire of commands. The free Flashlight app helps you find, install, and manage the necessary plugins.

[…]

Here are some examples:

  • Perform a quick Web search by inserting / before a Spotlight query.
  • Search Google Maps by prefixing your query with “google maps” or just “gm”.
  • Eject a mounted volume. You can also eject all mounted volumes by typing “eject all”.

[…]

You can install any of hundreds of plugins. The Flashlight app offers browsable categories including Design, Developer, Language, Media, and System, or you can search through the full set.

Looks like it works via SIMBL-style code injection, so I’m not sure how long it will last.

Update (2015-09-03): Nate Parrott (via Phil Dokas):

Hacking Spotlight is impossible in 10.11.

Every Swift Value Type Should Be Equatable

Andrew Bancroft:

To be truly equal, the == operator not only needs to be implemented, but it needs to be implemented in such a way that it behaves as we’d expect when doing our comparisons. During the talk, Doug mentioned three important properties of equality that need to hold for our Value Types:

  1. The comparison must be reflexive
  2. The comparison must be symmetric
  3. The comparison must be transitive

[…]

If Reference Types are involved with your Value Type implementation, things could get a little more complicated. “Complicated” probably isn’t the right word… but you do have to think a little more about your Type’s equality semantics.

Higher Order Functions in Swift 2

Anthony Levings:

Where you have a nested array that simply needs joining with an additional array of values placed between each use join(). If you have an array that needs to be flattened and needs additional values appended as a suffix or prefix to each nested array use flatMap(). If you require a one off initial value and possibly to append values before or after nested values use reduce().

[…]

The focus of this post has been arrays, but actually there are things that can be done with dictionaries as well.

Update (2015-07-27): Natasha Murashev:

In other words, flatMap is specifically overloaded to deal with Optionals. It will take an array of optionals and return an array of unwrapped optionals without any nils.

[…]

The idea of thinking of flatMap as dealing with containers vs just arrays makes things a lot clearer!

Update (2015-09-25): Giovanni Lodi:

A good use case for map and flatMap in the context of Optional is to simplify code using if lets.

North America Out of New IPv4 Addresses

Iljitsch van Beijnum:

ARIN, the American Registry for Internet Numbers, has now activated its “IPv4 Unmet Requests Policy.” Until now, organizations in the ARIN region were able to get IPv4 addresses as needed, but yesterday, ARIN was no longer in the position to fulfill qualifying requests. As a result, ISPs that come to ARIN for IPv4 address space have three choices: they can take a smaller block (ARIN currently still has a limited supply of blocks of 512 and 256 addresses), they can go on the wait list in the hopes that a block of the desired size will become available at some point in the future, or they can transfer buy addresses from an organization that has more than it needs.

[…]

The Internet Engineering Task Force (IETF) saw the eventual depletion of IP addresses looming in the early 1990s, so they set out to solve the problem and came up with a new version of the Internet Protocol. The old IP has version number 4; the new version is 6. IPv6 increases the length of IP addresses to no fewer than 128 bits—sort of like increasing phone numbers from 10 to 40 digits. As a result, the number of available IPv6 addresses is, for all practical purposes, unlimited.

The trouble is that, of course, old systems can only handle the IPv4 with its 32-bit addresses. That problem has pretty much been solved in the intermediate decade, and today virtually all operating systems can handle 128-bit IPv6 addresses—although some applications can’t or don’t handle them properly.

The main issue remaining is that most networks simply haven’t enabled IPv6 yet. Although turning on IPv6 is not as hard as some people think, it’s not entirely trivial either in larger networks.

Removing Previous Versions of Files

Dan Moren:

Reader Mike Barron points out you actually can remove all old versions: “You can remove all previous versions by simply holding the option button when navigating to File→Revert. When doing so you should see “Delete This Version…” change to “Delete Old Versions…” You will receive a confirmation before the deletions occur.”)

Tuesday, July 21, 2015

Differential Synchronization

Neil Fraser (via Mike Ash):

Keeping two or more copies of the same document synchronized with each other in real-time is a complex challenge. This paper describes the differential synchronization algorithm. Differential synchronization offers scalability, fault-tolerance, and responsive collaborative editing across an unreliable network.

[…]

Differential synchronization is a symmetrical algorithm employing an unending cycle of background difference (diff) and patch operations. There is no requirement that "the chickens stop moving so we can count them" which plagues server-side three-way merges.

[…]

Despite the inherent complexity, this synchronization system works extremely well. It is robust, self-healing and (with the proper diff and patch algorithms) impressively accommodating of users who are working on the same text.

Dictation Buffer Updates

Nicholas Riley:

It’s been a little over a year since I started using Dragon Medical in Windows as a dictation buffer for my Mac. Please see my previous few posts on the subject for some background.

Since then, I’ve eliminated pain points and added features which have made the dictation experience smoother and less infuriating. Once again, while this does not represent a turnkey system, in keeping with the DIY nature of such projects, hopefully it may help others out who want to do something similar. The code remains up to date on GitHub and I plan on maintaining it for my own use until something better comes along.

xonsh Shell

xonsh (via @UnixToolTip):

xonsh is a Python-ish, BASHwards-compatible shell language and command prompt. The language is a superset of Python 3.4 with additional shell primitives that you are used to from BASH and IPython. xonsh is meant for the daily use of experts and novices alike.

Embrace Cross-posting

Manton Reece (congrats):

When App.net was first taking off, many microbloggers struggled with how to decide where to post their short-form writing. Should they post some topics to Twitter and others to App.net? Should they cross-post everything to both services? At the time, there was an informal consensus that cross-posting was a cheat. It couldn’t take advantage of each platform’s strengths, and followers might often see the same post twice.

I now believe that cross-posting is a good thing.

I’m now posting blog links to both Twitter and App.net, and each blog post now links to its day on my Twitter timeline. I’m not sure about cross-posting conversations, though. I don’t like reading the same posts twice.

My Two Years as an Anthropologist on the Photoshop Team

Charles Pearson (via Hacker News):

Almost two years ago, the Photoshop team pivoted to focus its energies and resources on design features and workflows. To be successful, the team needed to understand trends in design and tools, as well as develop connections and empathy to design and designers. Worth noting, the pivot happened not long after Adobe moved to a subscription service and away from big box releases every 1–2 years. The subscription model provided an opportunity for development to be more iterative, but so much had to be re-thought, including research and customer feedback loops. This was the task then: build deeper knowledge and empathy around UI design, as well as develop feedback loops suited to new development cycles. As an anthropologist and ethnographer (the first ever at Adobe!), I was hired as a consultant to help address those gaps.

[…]

While Photoshop had fantastic feedback loops and relationships with its photography and imaging customers, there was no real conversation happening with designers. The connections were weak.

[…]

We heard things like: “The limitations of Photoshop aren’t with visual design. It’s with capturing the experience — that’s where the barriers are.” Meaning, it was difficult to work quickly and efficiently on a design meant for multiple devices and/or multiple screens. It wasn’t easy to recycle and repurpose elements, to create a responsive design system.

[…]

The Photoshop team is organized by “molecules” — small teams of design, engineering, and management that work on a particular feature. So for example, one molecule, called Cyan (real name), developed artboards. Via a private group in Slack, which turns out to be easiest way to keep conversations focused and tight, we matched up around 20 designers with members of the Cyan team.

Monday, July 20, 2015

Java Strings No Longer Share Storage

Heinz M. Kabutz (via Hacker News):

From Java 1.0 up to 1.6, String tried to avoid creating new char[]’s. The substring() method would share the same underlying char[], with a different offset and length. For example, in StringChars we have two Strings, with "hello" a substring of "hello_world". However, they share the same char[].

This is no longer the case with Java 7:

“Why this change?”, you may ask. It turns out that too many programmers used substring() as a memory saving method. Let’s say that you have a 1 MB String, but you actually only need the first 5 KB. You could then create a substring, expecting the rest of that 1 MB String to be thrown away. Except it didn’t. Since the new String would share the same underlying char[], you would not save any memory at all. The correct code idiom was therefore to append the substring to an empty String, which would have the side effect of always producing a new unshared char[] in the case that the String length did not correspond to the char[] length.

Apple’s GCD actually employs the opposite optimization: sometimes a larger data object shares the storage of two smaller ones, unless you specifically ask for contiguous storage.

stygiansonic:

So, it’s a trade-off between the original and new behaviour; the original way more or less caps the memory usage at the size of the original string, but at the expense of not being able to GC it if even a single substring exists, while the new way increases memory usage for each substring generated but does not prevent any of the strings from being GC’d.

This is why the article’s code example yields such a huge difference in memory usage in Java 6 vs Java 7; it is effectively a sort of "anti-pattern" when used against the new substring() method. (i.e. iterating through a large string and generating lots of sub-strings).

TheLoneWolfling:

The thing I mind is that there is now no way to get the old behavior. String is a final class, so you cannot override it and add a field, even. You can roll your own - if there is no code you do not control that takes a string. (And if you don’t mind having to write your own string class!)

mcherm:

I would not have objected to releasing this change in Java 1.7. But releasing it in a BUG FIX RELEASE causes me to lose confidence in the maintainers of the JVM. One of the reasons that large companies like mine build in Java is because of Sun’s long history of extremely careful attention to backward compatibility. Oracle is no Sun.

stygiansonic:

I agree - echoing the sentiments of another commentator here, I feel like one of the tenets of Java is backwards compatibility. While the change doesn’t affect functionality, it can turn code that previously had a space complexity of O(1) into one that is O(n). This is probably a Bad Thing.

bondolo:

I’m the author of the substring() change though in total disclosure the work and analysis on this began long before I took on the task. As has been suggested in the analysis here there were two motivations for the change;

  • reduce the size of String instances. Strings are typically 20-40% of common apps footprint. Any change with increases the size of String instances would dramatically increase memory pressure. This change to String came in at the same time as the alternative String hash code and we needed another field to cache the additional hash code. The offset/count removal afforded us the space we needed for the added hash code cache. This was the trigger.
  • avoid memory leakage caused by retained substrings holding the entire character array. This was a longstanding problem with many apps and was quite a significant in many cases. Over the years many libraries and parsers have specifically avoided returning substring results to avoid creating leaked Strings.

[…]

The comments about the substring operation becoming O(n) assume that the substring result is allocated in the general heap. This is not commonly the case and allocation in the TLAB is very much like malloca()--allocation merely bumps a pointer.

[…]

We investigated the regressions to see if performance was still acceptable and correctness was maintained. The most significant performance drop turned out to be in an obsolete benchmark which did hundreds of random substrings on a 1MB string and put the substrings into a map. It then later compared the map contents to verify correctness. We concluded that this case was not representative of common usage. Most other applications saw positive footprint and performance improvements or no significant change at all. A few apps, generally older parsers, had minor footprint growth.

cypherpunks:

Our importer went from a few minutes to parse a couple gigabytes of data to literally centuries. In the context of theoretical computer science that means correctness is preserved. In the real world, however, this means that the program stops progressing until a frustrated user presses the cancel button and calls our hotline.

[…]

It could have been so easy. Introduce a new function called something like subcopy(). Make substring() deprecated. In the deprecation comment, explain the memory leak problem and announce that substring() is schedule for removal in java 2.0. Port the jdk and glassfish and your other applications which might have a problem to use subcopy() everywhere when available. Check for performance regressions. Once java 2.0 is released, you can reclaim the memory for the offset and index variables.

And here is he crux of the problem: there is no java 2.0. The optimal time frame for making a set of major changes to the language has already passed, and nobody dares to propose it now. What you do instead is to release backwards incompatible changes anyway, as we see here, because you cannot fix all the old problems in any other way. This was already bad when upgrading between minor versions. Now we get the same in bugfix releases, and additionally, we need to look up some new bizzare numbering scheme to see which bugfix release is actually just fixing bugs and which isn’t.

FastMail Enables IMAP Push for iOS

FastMail (via Gabe Weatherhead):

While our own app has had push notifications for some time, with the built-in Mail app you would have to wait until it next decided to check the server for your new messages. But no more! From today, new mail will be pushed straight to your inbox.

robn_fastmail (via Milen Dzhumerov):

IMAP “push” usually means the IDLE extension, where a client holds a connection open and waits for the server to report that something has happened. This works ok, but isn’t great on mobile because holding a TCP connection open is usually difficult on flaky networks and consumes battery.

iOS Mail doesn’t implement it, instead doing a poll every 15 minutes. However it also implements a separate push system which allows true push if the server supports it. The details on how to support this aren’t public information.

We talked to Apple, and they were kind enough to give us access to this system. We implemented it on our side, and now when an email arrives at FastMail we can immediately signal to Mail.app that there’s new mail available.

Emphasis added.

Apple vs. the PC Industry

John Gruber:

To put Apple’s current industry position in perspective, the company probably sold somewhere between 60-65 million iOS devices last quarter. (I’m guessing ~50 million iPhones, ~10-12 million iPads, and a handful of million iPod Touches.) The average selling price of a PC has fallen to under $400. The average selling price of an iPhone has been estimated to be as high as $660. So while iOS devices, taken as a whole, might still fall a few million units short of the PC industry, they’re clearly generating more in revenue.

Photos for Mac 1.1

Jason Snell:

Yes, in Photos 1.1 you can add a location to an image or batch of images that weren’t geotagged, as well as edit the location of data of already-geotagged images.

Nick Heer:

Based on what I’ve heard and played with so far, I don’t think that I’m going to fall in love with it yet, in the same way I did with Aperture. But it’s now the primary way I edit my photos, and I like it more each time I use it.

[…]

As he pointed out, it relies on the Apple Maps POI database, which can be a crapshoot as we’ve previously discussed. Furthermore, because it relies upon search, it’s incredibly difficult to bulk tag photos in slightly different places – that is, you must tag them all identically, or modify them one at a time, which is tedious.

I don’t see anything there about using a GPS track to assign locations to photos.

I’m in the progress of learning Lightroom. It is quite a shift from Aperture, but I am liking a lot of what I see. And, unlike Photos, it supports both Google Maps and terrain view.

Friday, July 17, 2015

App Transport Security

WWDC 2015 Session 706 (video, PDF):

If you link your app against OS X El Capitan or iOS 9, by default it won’t be able to make any unprotected HTTP connections; they will simply fail, and any TLS connections that your app makes will have no adhere to these best practices that I just described. And that’s TLS 1.2 with forward secrecy. Cryptographic primitives that are known to be insecure are not allowed, and there are minimum key size requirements put in place.

Now, you may not be able to comply with all of these restrictions immediately, and that’s okay, you can specify exceptions in your Info.plist either on a case-by-case basis for each domain or as a global override. But, as an industry we have to very soon get to a point where no user information hits the network unencrypted, and we need your help.

What’s New in iOS 9:

App Transport Security (ATS) lets an app add a declaration to its Info.plist file that specifies the domains with which it needs secure communication. ATS prevents accidental disclosure, provides secure default behavior, and is easy to adopt. You should adopt ATS as soon as possible, regardless of whether you’re creating a new app or updating an existing one.

If you’re developing a new app, you should use HTTPS exclusively. If you have an existing app, you should use HTTPS as much as you can right now, and create a plan for migrating the rest of your app as soon as possible.

WWDC 2015 Session 711 (video, PDF):

Essentially all you have to do is declare which domain needs to load over HTTP and NSURLSession will still allow that cleartext load.

Now we also know that some of you may be supporting a general purpose web browser or another app that loads URLs based off dynamic user input, and we’ve allowed for that case.

[…]

So, do note that ATS is only active if you build against a current SDK.

[…]

NSURLSession will transfer all HTTP resources or loads URLs to HTTPS automatically.

App Transport Security Technote:

All secure http (https) connection follow the App Transport Security default behavior in apps built for iOS 9.0 or later, and OS X 10.11 or later. Connections that do not follow the requirements will fail.

[…]

You can specify exceptions to the default behavior in the Info.plist in your app or extension. Keys in the plist are used for specific exceptions or to turn off App Transport Security. Table 1-1 shows the keys and their types.

Blake Gentry (comments):

The ideas behind App Transport Security are great. It’s essentially HTTP Strict Transport Security for apps, making it much harder for developers to inadvertently disclose private user information. The feature that will benefit the privacy and security of millions of Apple customers. The writing is also on the wall that Apple intends to make this feature mandatory at some point, essentially deprecating plaintext HTTP altogether.

Apple, however, has yet to take their own advice. There are many OS X components and Apple apps that still do not use encryption exclusively, relying on HTTP over port 80. Here’s an example from the brand new Photos app, communicating with AWS S3 over port 80 […] So far I’ve encountered 9 separate OS X services or first-party apps that are still relying on plaintext HTTP.

See also: The Rush to “Deprecate” HTTP.

FairPlay Streaming

Chris Adamson:

Because here’s the difference — and you figure there has to be a difference for it to be worth Apple’s time to develop an supplementary standard to what they already have and everyone’s using. The point of “FairPlay Streaming” seems to be to plug holes in the server-to-viewer chain where the media could be copied in some way.

[…]

The standard seems particularly interested in preventing you from AirPlay’ing content to other devices.

[…]

So the first part here is that when AirPlaying to an Apple TV, FairPlay Streaming basically turns the Apple TV into a Chromecast: it transfers responsibility for downloading and playing the stream to the AppleTV, rather than having the iOS/OSX device fetch the content and then re-stream it to the Apple TV. Good in one sense, because this effectively halves the amount of data being sent across your network.

But look at that last point: FPS content cannot be captured by the user if they’re, say, streaming it to their Mac. It’s not clear if this is also the behavior when playing an FPS stream directly in OS X or iOS. It’ll suck if it is, because this basically would mean that streams would behave the same way DVDs do when you try to get a screenshot: you get a blank rectangle.

[…]

So this seems like a feature that’s likely to do little to prevent piracy, while taking away the ability to do things we already do, like grab screenshots, livestream group viewings, or screen-record things like reaction videos. And even if it works, we’ll just seek out workarounds[…]

Hardware Audio Encoding, AVMovie Brings Back QuickTime Features

Chris Adamson:

We’ve found we have to fall back to the software encoder because the hardware encoder (kAppleHardwareAudioCodecManufacturer) would cause ExtAudioFileWrite() to sometimes fail with OSStatus -66570 (kExtAudioFileError_AsyncWriteBufferOverflow). So I asked about that and was told “oh yeah, we don’t support hardware encoding anymore… the new devices don’t need it and the property is just ignored”. I Slacked this to my boss and his reaction was “would be nice if that were in the documentation!”

[…]

Enter Tim [Monroe], and the new AVMovie. This is a new editing API, an alternative to the AVComposition stuff, and is much more strongly influenced by the old QuickTime APIs (and, in turn, by the QuickTime file format itself). This brings back some of the important concepts of the old QuickTime API, such as modifier tracks: tracks that consist only of instructions to modify other tracks at certain times. AVComposition offered some support for this with opacity ramps and affine transforms (in the hideously-named AVMutableVideoCompositionInstruction class) but QuickTime was always able to offer a much broader range of functionality, and could save those instructions in the file itself until they were needed for a final export.

[…]

And yes, I did say “headers”. The new QuickTime-inspired editing APIs aren’t in the OS X 10.11 docset yet, and they don’t exist at all on iOS.

I’m certainly happy to see this new approach to editing video, although it’s going to be hard to clarify when to use AVMovie versus AVComposition. For example, doing compositing work with affine transforms and opacity, or pulling in Core Animation graphics, may work better with the AVComposition APIs, at least until AVMovie is built out and understood. And for the hardcore, there’s always the low-level approach of doing your own rendering, perhaps with Core Image, and writing samples with AVAssetWriter.

[…]

I wonder about the internal design and architectural politics that led to this about-face on AV Foundation editing, after holding to the company line for so long on the AVComposition stuff.

Evan Williams on Twitter Third-party Devs

Manton Reece:

Just acknowledging this publicly is progress. It’s now been almost 3 years since my last tweet. I don’t expect to return to Twitter, but I’m still very interested in whether there’ll be a noticeable change in direction from the new CEO.

Update (2015-07-20): The source of the quote is here.

Security-Scoped Bookmarks and .webarchive Files

Matthias Gansrigler:

The bug itself is described fairly easily. You have a .webarchive file you’d like to move using Yoink. So you drag it onto the app and then move it from Yoink to the actual destination. Business as usual.

Only that now, instead of opening the webarchive in your standard browser when double-clicked, above’s warning is shown.

[…]

There are two ways a security-scoped bookmark can be created – either with read and write access, or read-access only (NSURLBookmarkCreationWithSecurityScope or it plus NSURLBookmarkCreationSecurityScopeAllowOnlyReadAccess).

I don’t know why this causes the issue, but changing the bookmark creation options from read and write access to read-access only fixed the issue (and moving the file is still possible with Yoink). It definitely looks like a sandbox bug to me.

Why should an app-internal bookmark (mind you, it’s only a reference to a file, not the file itself, created from an NSURL object) write to and change the webarchive’s file so that it a) can’t be opened anymore and b) shows the bookmark-creating-app as its creator?

Minor correction: all the .webarchive files I’ve seen are binary property lists, not bundles.

UI Testing With Xcode 7

Laurin Brandner:

About two weeks ago, Apple released the first beta of Xcode 7 which comes with shiny new features like built-in UI testing. Apart from Swift, this was easily the feature that excited me the most. I couldn’t wait to try it out. Not that UI testing is anything new; KIF has been around since 2011. It’s a great tool but it has its limitations: it’s slow, it’s not 100% reliable and it depends on private API. Therefore, an official solution seemed like a good idea.

[…]

Because Xcode 7 was constantly crashing, I had to decrease the number of iterations to 5 times and then multiplied it by 4.

[…]

XCTest makes it easy to write simple UI tests. More advanced tests are difficult or just not possible. KIF is a lot more flexible.

Compared to KIF, however, XCTest is slightly faster at finding the UI elements. With animations disabled, XCTest is really fast.

Thursday, July 16, 2015

Unpacking Git Packfiles

Aditya Mukerjee (via Hacker News):

I’ll count packfiles as the third strategy that Git uses to reduce disk space usage, even though packfiles were really created to reduce network usage (and increase network performance). It’s helpful to keep this in mind because the design of Git’s packfiles were informed by the goal of making network usage easy. Reducing the disk space needed is a pleasant side effect.

[…]

The packfile starts with 12 bytes of meta-information and ends with a 20-byte checksum, all of which we can use to verify our results. The first four bytes spell “PACK” and the next four bytes contain the version number – in our case, [0, 0, 0, 2]. The next four bytes tell us the number of objects contained in the pack. Therefore, a single packfile cannot contain more than 232 objects, although a single repository may contain multiple packfiles. The final 20 bytes of the file are a SHA-1 checksum of all the previous data in the file.

The heart of the packfile is a series of data chunks, with some metainformation preceding each one. This is where things get interesting! The metainformation is formatted slightly differently depending on whether the data chunk that comes after it is deltified or not. In both cases, they begin by telling us the size of the object that the packfile contains. This size is encoded as a variable-length integer with a special format.

[…]

While it’s possible to to work around the aforementioned buffering issues and parse a packfile without ever reading the IDX file, the index makes it a lot easier. Like the packfile, a version 2 index file starts with a header, though the index file header is only eight bytes instead of 12. […] After the header, we encounter what Git calls a fanout table.

Aditya Mukerjee:

I discovered this while working on a clean-room implementation of Git in pure Go. While there are a lot of references to packfiles online, surprisingly, the actual format of packfiles was rather underdocumented. Most resources just mention that they exist, and describe how to use git verify-pack to inspect a packfile, without explaining how to parse packfiles and apply deltas.

I decided to write this up to save others the trouble of having to reverse-engineer it from scratch!

Obergefell v. Hodges: the Database Engineering Perspective

Sam Hughes (epic 2008 post):

Altering your database schema to accommodate gay marriage can be easy or difficult depending on how smart you were when you originally set up your system to accommodate heterosexuality only. Let’s begin.

[…]

No matter how advanced and flexible your table structure, it will always be possible to create data which cannot fit into it. At that time, you will need to change your database. And the longer it’s been since you did, the less pleasant that’s going to be.

The lesson is not “prepare for every possible eventuality”. The lesson is to become comfortable and confident in modifying your schemata without losing data, and rolling back botched changes. Do this regularly, so that it becomes second nature. The lesson is to get used to change.

And what is true of our databases is also true of our world views. The future is vast and humans are creative. Things are going to happen which nobody could predict.

Sam Hughes (2015 update):

To investigate the specific ramifications of today’s ruling, however, here’s the schema we’re probably starting with:

[…]

Already the constraints on a schema like this are quite complicated. husband_id and wife_id are both foreign keys for column people.id. Check constraints ensure that the value of marriages.husband_id always points to a people row with gender set to “male” and the value of marriages.wife_id always points to a row with gender set to “female”. (Exactly how the gender column should be structured is outside the scope of this essay, but the values “male” and “female”, at least, should be available. Structuring the name column is even further out of scope, because yikes.) divorce_date is nullable. Probably there ought to be another check constraint which ensures that divorce_date doesn’t come before marriage_date.

It might be required to incorporate some sort of check for duplicate combinations of husband_id and wife_id… but then again, this could make it impossible for a couple to e.g. marry in 1994, separate in 2009 and then remarry in 2015.

[…]

But the more interesting thing is that you just incidentally let in a whole bunch of edge cases. Up until now, it wasn’t possible for an individual to marry themself. Now it is, and you need a new check constraint to ensure that partner_1_id and partner_2_id are different. Regardless of concerns about duplicate rows/couples remarrying, you also now have to contend with swapped partners: Alice marries Eve, and also Eve marries Alice, resulting in two rows recording the same marriage. This can typically be prevented by ensuring that partner_2_id is greater than partner_1_id, which would incidentally also prevent self-marriage as described above. Note that this could in turn invalidate previously-existing heterosexual marriages where the husband_id was lower than the wife_id. This constraint would have to be applied for future inserts only, or the disordered rows would need to be swapped.

Feeder 3.0

Steve Harris:

It’s now possible to share Feeder’s library with cloud services such as iCloud Drive and Dropbox by placing the library folder in the appropriate location, thanks to a new library format — there is a new Move Library command in the Feeder menu to do that for you. Feeder automatically updates as soon as it detects a change has been made.

[…]

With this release, I have decided to stop offering Feeder for sale on the Mac App Store, so can offer the same upgrade deal to all customers, along with the fastest updates and best service I can possibly provide.

See also: BBEdit Leaving the Mac App Store.

Update (2015-08-14): Steve Harris (comments):

Even though not all my sales go through the App Stores, Apple’s 30% cut far exceeds what I pay the UK government in Income Tax and National Insurance each year, and for that I get things like healthcare, pension, education, transport, emergency services, defence, etc. To think of it another way, if I add up all the money they’ve taken since the store’s launch in 2011, it could pay my rent for almost 7 years.

FastSpring, who process my direct sales, take 10%. They don’t promote or review the apps, host downloads and so on, but they do handle things like regional sales taxes and allow the developer to know who their customers are, process refunds, etc. Developers in business before the Mac App Store know firsthand that you can do it cheaper, with more control and flexibility AND provide better service to your customers by selling your apps yourself.

Steve Harris:

Following my post earlier in the week on how Apple’s 30% cut of all Mac App Store sales is threatening the very existence of this business, I have decided to take action and introduce some transparency to the pricing. If Apple wants a 30% markup to everything they sell through the Mac App Store, that should be obvious.

Yes, Apple does allow the Mac App Store price to be higher than the direct price.

Cmd-Number Shortcuts for Safari 9

Daniel Jalkut:

In Safari 8 and earlier, keyboard shortcuts combining the Command key and a number, e.g. Cmd-1, Cmd-2, Cmd-3, would open the corresponding bookmark bar item. So if you arranged your most-frequently-visited sites in the first few bookmark bar slots, you could easily jump to those pages by muscle memory thanks to these shortcuts.

In Safari 9, these shortcuts now switch to any open tabs you have in a Safari window. This will come as a surprise to folks who have gotten used to e.g. using Cmd-1 to quickly jump to e.g. Google News, or Yahoo Stocks.

The implicit shortcuts for bookmark bar items are still available, but you have to add the option key into the mix. So where you used to press Cmd-1, you must now press Cmd-Opt-1.

Wednesday, July 15, 2015

Apple Watch and Water

Craig Hockenberry (comments):

Make sure you rinse your equipment in fresh water after it has been exposed to salt water. As you’ve seen above, that includes a swimming pool.

[…]

The biggest problem with the Workout app is that it’s basically unusable while you’re in the water. As we’ve learnt, both the touch and force press controls stop working. There’s no way to pause the workout. You have to start the workout before getting in the water and stop after you’ve gotten a chance to dry off.

For an ocean swim, this screws up your timing: you don’t really start swimming until after you get beyond the surf break. If there’s heavy surf, this can take several minutes. Workout data is being collected while you’re standing and waiting for waves to clear.

[…]

The elephant in the room: the touchscreen doesn’t work reliably anywhere near water. The source can be the ocean, a swimming pool, or your own sweat. At the same time, the Workout app is heavily dependent on touch[…] I’m convinced Apple’s recommendation to not use the watch in water is because of the erratic behavior it causes.

Dr. Drang:

The 316L stainless steel alloy Apple uses in the Watch is, as Greg Koenig said in this iMore article, quite resistant to corrosion. Your Apple Watch is extremely unlikely to develop corrosion pits or stains. But stress corrosion cracking, despite the word corrosion in its name, is a distinctly different phenomenon. SCC can break an object apart even as its surface remains bright and shiny.

[…]

316L is known to be susceptible to SCC in a chloride environment, like salt water or salt spray, so two of the three requirements for stress corrosion cracking, material and environment, are met. What about stress?

[…]

It is, I think, on the stress side that Apple is preventing SCC. By keeping the residual stresses low—or by making sure the residual stresses at the surface are compressive rather than tensile—Apple is eliminating one of the three requirements for stress corrosion cracking, and that’s why Craig Hockenberry’s Apple Watch won’t split open, even if he goes swimming in the Pacific every day.

How iCloud Drive Deletes Your Files Without Warning

Mark Jaquith (via Hacker News):

And let’s say that, on your shiny new Mac, you want to move these files from iCloud Drive to your local hard drive, or to another synced drive like Google Drive or Dropbox. Well, you can just drag their folders do the other destination, right? You sure can. Apple kindly warns you that your dragging action is moving that folder, and that the files will be moved to your Mac, and won’t exist on iCloud Drive anymore. Fine. That’s what dragging a file from one place to another generally does!

But what happens if there are files inside this folder that haven’t yet synced to your local machine? Well, the move operation will be slower, because your Mac has to first download them from iCloud Drive. But once they download, they’ll be in their new location. Right?

Nope. Those files are now gone. Forever.

In their place, is a file named FILENAME.original-extension.icloud. This file, only a couple kilobytes in size, is the placeholder that OS X uses to pretend that the file existed on your system. Your original file is gone. It’s gone from iCloud Drive, and it exists nowhere on your hard drive.

Cloud syncing is not a backup, and iCloud (still) doesn’t have good failure modes.

iCloud Drive FAQ (via eclipxe):

If you need to access a file that you recently deleted, you might be able to recover it from iCloud.com. Sign in to iCloud.com, click Settings > Data & Security, then browse the list of files in the Recover Documents tab. Files will be removed from Recover Documents in 30 days.

This is non-obvious and seems to not preserve the folder hierarchy, which could be a serious problem, but it’s better than nothing. In my case, it showed a bunch of unsaved documents that I had discarded.

nickbauman:

Apple doesn’t have the right organizational structure to support cloud services across apps to a level of consistency and quality that Google and Amazon does. Nathan Taylor explains this quite well.

jsmthrowaway:

Speaking as a former resident deep in Cue’s org, I believe this is common knowledge internally, but there isn’t alignment on how to address it. Even at the IC level, most people I spoke with are aware of Apple’s shortcomings in services. Services are not the favored child at Apple. iOS and hardware are, because of revenue. Services ICs know that, and it hits morale directly; I saw (and felt) this. There are attempts to fix it, too, but those manifest as reorganizations. I was subjected to four in a year and a half.

You hear pains from teams like Maps, who were moved from iOS to services during my tenure, and who immediately ran into serious organizational problems, dried up budgets, and so on. The gettin’ is good in iOSville, and once you leave iOS, it’s a whole ’nother Apple. There’s a common story about the origin of Maps at Apple where Maps was basically given a blank check, and they’re still mopping up some of that excess to this day. That doesn’t happen in services.

Meanwhile, organizationally, Siri is kind of outside the typical services structure for various legacy reasons and they’re off iterating like all getout and having a blast without the encumberance of the services organization. Every time I met with Siri I always came away with questions like, in this organizational climate, how on earth are they getting so much done?

Apple needs a serious Microsoftism on services. If you would have told me five years ago that Microsoft under Nadella would completely reverse course and embrace the living hell out of services while Apple meandered in the “let’s buy companies to implement our services strategy” grasslands, I’d have said the opposite is more likely, yet here we are.

Update (2015-07-16): Dan Moren:

[…] back in the iDisk era I encountered a similar data loss loophole.

Adobe Replaces “Rank and Yank” With “Check In”

Anne Fisher (via Hacker News):

Especially troublesome was that the company’s “rank and yank” system, which forced managers to identify and fire their least productive team members, caused so much infighting and resentment that, each year, it was making some of the software maker’s best people flee to competitors.

So, based in part on ideas crowdsourced from employees, Morris and her team scrapped annual evaluations and replaced them with a system called Check In. At the start of each fiscal year, employees and managers set specific goals. Then, at least every eight weeks but usually much more often, people “check in” with their bosses for a real-time discussion of how things are going.

[…]

Morris says that transparency has paid unexpected dividends. For one thing, fewer valued staffers are leaving, despite the ferociously competitive Silicon Valley market for tech talent.

Perfect App Store Screenshots

Felix Krause (via John Saddington):

All MindNode screenshots shown above are created completely automatically using 2 steps:

Using snapshot you can take localised screenshots on all device types completely automatic. All you have to do is provide a JavaScript UI Automation file to tell snapshot how to navigate in your app and where to take the screenshots. More information can be found on the project page. This project will soon be updated to use UI Tests instead of UI Automation to write screenshot code in Swift or Objective C instead.

This step will create the raw screenshots for all devices in all languages. At this point you could already upload the screenshots to iTunes Connect, but this wouldn’t be so much fun.

[…]

frameit was originally designed to just add device frames around the screenshots. With frameit 2.0 you can now add device frames, a custom background and a title to your screenshots.

MAIKit: Framework for Sharing Code Between iOS and OS X

Michael Donald Buckley:

Third-party frameworks like Chameleon exist, but because UIKit lacks APIs for some features of OS X, apps built with UIKit feel limited on OS X. However, since UIKit and Appkit are very similar, many developers alias UIKit and Appkit classes and use methods common to both frameworks.

For the past few months, I've been experimenting with automating this technique by parsing the UIKit and AppKit headers to extract the interfaces common to both, and the result is MAIKit. I thought that even if this wasn't an ideal way to develop cross-platform code, if this is something a lot of developers are doing, we could all benefit by automating the process.

“MAI” stands for “Mac and iOS.”

Font Parsing Vulnerabilities

John Villamil:

During my research into various OS X frameworks I chose to focus on OS X font parsing and spent a week fuzzing and reversing native libraries. This research resulted in six CVEs, five of which are shared between OS X and iOS.

Client side font parsing is often a good target because the file formats are varied and complicated. For example, TrueType comes with its own turing complete instruction set which you can learn more about here. OTF and the less popular PostScript file formats are complex and also supported.

Many of these flaws are the result of using untrusted length values extracted directly from the file without validation. In one example CoreText, a low level font layout framework, the ArabicLookups::AddLookup function (shown below) reads a length value from the memory mapped font file, using it to increment a pointer out of bounds. The pointer is held in the rdi register which is later dereferenced in the ResolveLookup function.

iPod touch 6th Generation

Jim Dalrymple:

Apple also made some significant changes to the cameras in the iPod touch. In addition to an 8 Megapixel camera, Apple updated the image sensor processor and added burst and slo-mo modes to the iPod touch.

Not to be outdone by the iPhone when capturing movies, the iPod touch also gains cinematic video stabilization. This is especially handy if you’re capturing a movie while moving—the technology stabilizes the movie automatically.

If, for some reason, you don’t want an iPhone, this seems like a great deal for a fast iOS device with a solid camera.

Jason Snell:

The new iPod touch has the same weight and dimensions as the previous model, and continues to sport a 4-inch Retina screen. (Yes, this means that for the moment, the new iPod touch is the most advanced device Apple makes with a four-inch screen, since the iPhone 5S still uses the older A7 processor. This move also makes me even more convinced that Apple will release an A8-powered 4-inch iPhone this fall, as a successor to the 5S.)

One can hope.

Update (2015-07-16): John Gruber:

Word from a few little birdies is that what remains of the iPod software team is now working on Apple Watch — the Nano UI wasn’t updated to look like iOS 7 because there’s no one left to do it.

Update (2015-07-20): Matt Birchler (via Marco Arment):

With this week’s update to the entire iPod lineup, many have been asking “Who are iPods even for these days?” Well, I worked the last 3 years managing an electronics department for Target, and have sold a lot of Apple devices over that time. Since Apple doesn’t break down demographics for who is buying each device, I thought I would share my experience.

Rene Ritchie:

The iPod touch 6 is clearly better than the iPod touch 5, about the same as the iPad Air 2, and just behind the iPhone 6.

Update (2015-07-22): Kirk McElhearn:

So why buy an iPod touch, when I have an iPhone and an iPad? My use case is different from that of most people: I need a device I can use for testing, on which I can install beta software. The 5th generation iPod touch lagged a lot with the iOS 9 betas, and the new model is really fast; faster than my iPhone 5s. But beyond that, I plan to use the iPod touch as an Apple Music device. With all the problems caused by the iCloud Music Library, I won’t trust my iPhone, or my main iTunes library, to use this service. Having an iPod touch makes it easy to use Apple Music at home, in conjunction with a library on my MacBook Pro.

Tuesday, July 14, 2015

WebKit Hacking From the Bleeding Edge

Daniel Jalkut:

As an open source project, I initially believed I could build and run WebKit wherever I choose. After all, isn’t that what “open source” is supposed to be all about? But ah, there’s a catch. At least when it comes to building and running WebKit on OS X releases, there is a dependency on a small, binary-only static library which provides key system-specific linkages to WebKit. Usually this binary is added to the open source project around the time the system release goes public, but not much sooner.

The long and short of it? If you want to build WebKit and you don’t work at Apple, you need to do so from publicly released versions of OS X.

Python 3.5: async and await

What’s New in Python 3.5 (via Hacker News):

[PEP 492] added dedicated syntax for declaring coroutines, await expressions, new asynchronous async for and async with statements.

Third Hacking Team Flash Zero-Day Found

Brian Krebs:

For the third time in a week, researchers have discovered a zero-day vulnerability in Adobe’s Flash Player browser plugin. Like the previous two discoveries, this one came to light only after hackers dumped online huge troves of documents stolen from Hacking Team — an Italian security firm that sells software exploits to governments around the world.

News of the latest Flash flaw comes from Trend Micro, which said it reported the bug (CVE-2015-5123) to Adobe’s Security Team. Adobe confirmed that it is working on a patch for the two outstanding zero-day vulnerabilities exposed in the Hacking Team breach.

We are likely to continue to see additional Flash zero day bugs surface as a result of this breach. Instead of waiting for Adobe to fix yet another flaw in Flash, please consider removing or at least hobbling this program.

James Vincent (via John Gruber):

Alex Stamos, the recently appointed chief security officer at Facebook, has called on software company Adobe to announce an “end-of-life date for Flash.” In a pair of tweets sent over the weekend, Stamos echoed a number of recent complaints from the security community that the software has become the vector for just too many hacking vulnerabilities.

AppleEventBridge: Native AppleScripting Support for Swift

Hamish Sanderson:

In light of OS X 10.11 addressing some longstanding deficiencies in NSAppleEventDescriptor, I’ve been dusting off a fork of my old objc-appscript project, now renamed AppleEventBridge, modernizing and extending it both to take advantage of improvements to ObjC in the last few years and to add native support for Apple’s new Swift language:

https://bitbucket.org/hhas/appleeventbridge/

Unlike Apple’s flawed Scripting Bridge and JavaScript for Automation, AEB aims to provide application scripting capabilities and compatibility that equal (if not better) AppleScript’s own, along with superior documentation and developer tool support.

92% of Smartphone Profits

John Gruber:

At just 20 percent of unit sales, Apple isn’t even close to a monopoly. At 92 percent profit share, they have a market dominance that rivals any actual monopoly the tech industry has ever seen. We don’t even have a term for this situation, it’s so unusual.

Graham Lee:

That 92% profit on 20% sales is indicative, rather than contraindicative, of a monopoly. And there’s another word we could use, too: monopsony. Let’s say that you’ve made an iOS app, and now you want to sell it. Do you create a storefront on your website to do that? Do you contact Sears and see how many boxes they want? Speak to some third-party distributor? No, you can only sell to Apple, they are the only buyer for iOS apps.

[…]

I would imagine that a legal system that did explore this question would consider analogous environments, like the software market of the 1990s. Back then, Microsoft bundled a web browser and a media player with their operating systems and used their market power (which let them act as a monopoly even though competitors existed) as an operating system vendor to make it hard to sell competing browsers or media players. It might be an interesting thought experiment to compare that situation with today’s.

On Negative App Store Reviews During Betas of iOS and OS X

Federico Viticci:

I don’t know what the solution is, but I’ve been observing this problem for six years now, and I’d like to offer a list of suggestions and ideas that will hopefully act as reminders for the future.

[…]

We all have to keep in mind, though, that developers get the short end of the stick here. When it comes to App Store reviews that point out issues on betas of iOS and OS X, there is nothing they can do. They can’t respond to them, they can’t release compatibility and feature updates for public betas, and yet they’re left dealing with the outcome of negative reviews. These are smart folks, and they know that their apps have issues on beta versions of iOS and OS X. Not only it’s not useful to leave negative reviews for those problems now – it’s not fair to developers.

Monday, July 13, 2015

Contacts Framework Identifiers

Greg Pierce:

I’ve experimented with the the Contacts Framework a bit. One thing that caught my attention was that the documentation specifically calls out that identifier properties for contact, group and other objects vended by the framework can be persisted between app launches.

[…]

It appears the identifiers given are unique to the device, but not the app, and are changed if a contact is ever removed and restored to the device via any sort of sync process.

[…]

Long story short, it would not be practical for Drafts to have an action that linked directly to a contact, because it would not be possible to sync that action across devices and still have it point to the same contact.

That’s a shame. With the old Mac Address Book framework, there was a uniqueId (kABUIDProperty) property, a string that I think was the same across devices. (Unfortunately, contrary to the documentation, iCloud syncing could cause this permanent identifier to change.)

iOS instead had a typedef int32_t ABRecordID that also changed with syncs:

Every record in the Address Book database has a unique record identifier. This identifier always refers to the same record, unless that record is deleted or the data is reset. Record identifiers can be safely passed between threads. They are not guaranteed to remain the same across devices.

The recommended way to keep a long-term reference to a particular record is to store the first and last name, or a hash of the first and last name, in addition to the identifier. When you look up a record by ID, compare the record’s name to your stored name. If they don’t match, use the stored name to find the record, and store the new ID for the record.

In the new Contacts framework, the identifier is a string (like the old Mac API) that is only unique on a given device (like the old iOS API).

I assume that this was done for privacy reasons—though the benefit is not entirely clear to me—but it makes for more coding work to get an inferior user experience, with some features like Pierce suggests not really possible.

Update (2016-03-18): Greg Pierce:

Let’s talk about ContactsUI. Deprecate its predecessor and fix 0 bugs in it since iOS 9b1.

Sunday, July 12, 2015

Apple Music: Connect

Dave Wiskus:

Someone asked why I believed that Connect would ever be better than Ping, Apple’s previous attempt at socialifying iTunes. Ping’s mistake was that it tried to connect listeners to each other, as a way of discovering new music. Apple Music has re-thought that problem in some very interesting ways, and early indications are that the new approach works. For the social component, Connect wants to be about connecting artists with their listeners, but at the moment, it falls short.

Nick Heer:

Based on Wiskus’ documentation, it looks like it lacks the litheness of Twitter, the scale and engagement of Facebook, and the demo tape feel of SoundCloud. I don’t quite know what to make of it yet.

John Gruber:

Why not let artists pick songs from iCloud Drive, for example? Having to sync via iTunes on your Mac to post from your iPhone is so 2008.

Manton Reece:

If Apple Music can be thought of as Beats Music 2.0, then the Connect tab is probably a little like Ping 2.0, an update on Apple’s first attempt to build a music-only social network. As Daniel and I discussed on Core Intuition 187, any service that demonstrates a network effect — everything from eBay to Twitter — needs some critical mass of users to reach its potential. I was curious whether Apple could achieve this if the Connect feature was locked behind a paid subscription after the initial 3-month trial.

What I missed is that Connect and even Beats 1 will be free.

Jonathan Poritsky:

Artists postings thus far have been less than stellar. I think Apple has made a massive mistake billing Connect as a place to follow musicians. Connect is actually a wonderful service being squandered by Apple. The things that Apple is expecting artists to post just aren’t that interesting. Links to their own music and original photos or videos are relatively weak sauce, and the posts have been few and far between for most artists.

However, Connect is great for sharing exactly what I came to the Music app for: music. The trouble is most artists aren’t posting music; they’re promoting themselves in a fairly bland manner. I’ve found the best people to follow are DJs and performers with shows on Beats 1, as well as Apple’s in-house “curators.”

[…]

Right now anyone can share playlists through Apple Music. However, not anyone can get an account that can be followed. When that happens, I think Connect will be, unabashedly, the best music social network on the planet. The trouble right now is that Apple wants it to be a “a place where musicians give their fans a closer look at their work, their inspirations, and their world.” That’s not an interesting enough proposition.

Nick Heer:

By default, iTunes Connect auto-follows all the artists in your music library, regardless of how much you listen to them, whether they’re on a compilation, or whether you even want to see posts from them. For example, because of that whole U2 fiaso, you’re probably automatically following them, too. Turning that behaviour off is a little hidden, as Steven Troughton-Smith and Anthony F Waller found.

Update (2015-07-20): Kelly Hodgkins:

Connect isn’t for everyone, and you can remove it from Apple Music if you’re not interested. Once removed from Apple Music, the Connect tab is conveniently replaced by Playlists. Here’s how to make the switch.

Update (2015-08-21): Mitchel Broussard:

Following an interview yesterday with Evening Standard, Wired today posted an interview with Apple Music executive Jimmy Iovine, in which the Beats co-founder admits the company’s need to work to make Connect a better platform for artists and fans alike, and even hints at a possible curation aspect for Apple TV, similar to that of the company’s new streaming music service.

Apple Music: For You

Jim Dalrymple:

Many of the problems I had before with iTunes Radio are completely gone. Selecting genres of music or even something like “70s Rock Hits” or “80s Metal” gives you exactly what you want—great music.

With the integration of Beats, you also get curated playlists and the ability to stream artists’ music, if you become a member of Apple Music. The selection went from not having much to choose from with iTunes Radio, to having so much great content from playlists and radio stations that I had to start saving them all so I could listen to them later.

Dr. Drang:

This is not what I want from For You. I already have my musical brain sitting between my ears, and I have a 10,000-song representation of it in my iTunes library. But there are 30 million songs in Apple Music, and even if a huge swath of them can be eliminated because I hate Christian rock or speed metal or whatever, there will still be millions of songs left. For You should be my guide into the unexplored areas of those millions of songs. Right now, it’s not.

Christina Warren:

It’s hard for me to over-stress how much I like For You. From the very beginning, the recommendations in playlists and albums that the app showed me were dead-on accurate, reflecting my various musical interests.

Kirk McElhearn:

What’s also odd is that three of those albums are already on my iPhone; that’s not trying very hard.

Apple knows a lot about my musical tastes. It knows what I’ve bought from the iTunes Store, and it knows what’s in my iTunes library through its Genius feature. So it should get a lot better than that.

Khoi Vinh:

The playlists include a whole bunch of stuff I can’t stand, along with a smattering of albums from acts that I’m okay with but not particularly passionate about, and one so-so album from a band I quite like but rarely listen to. Nothing from my current heavy rotation of artists appears here, and nothing new or surprising that I’d never encountered before does, either. Overall, the selection lacks any real surprise or inspiration.

[…]

Apple does, in fact, have all the data that they need to make great recommendations to me. I’ve been using their iTunes Match service for two years, which stores all of my iTunes library’s metadata in the cloud. Moreover, I’ve been using iTunes for almost a decade and a half now, migrating the same music library from computer to computer and device to device. Few other services have as much data on my media consumption habits as Apple does—not Netflix, not Amazon, not Google. Plus, Apple is the most successful technology company in human history. When you combine those factors, shouldn’t it be realistic to expect more from Apple than what is basically the same out-of-the-box experience that the original Beats app offered?

Ben Lovejoy (via Nick Heer):

I gave my first impressions of Apple Music on day two, and my main disappointment remains: despite putting both owned and streamed music into a single app, there is absolutely no real integration between the two. All the evidence suggests that Apple Music has no awareness of my owned music.

I’ll get past that in a moment, but bear with me first for a couple of paragraphs. Because this is, in my view, more than just a missed opportunity: it’s almost criminally negligent. iTunes knows more about my musical tastes than my girlfriend. More than my neighbours, who have sometimes been more familiar with my musical tastes than they might wish. More than any of my friends – even the one who kindly ripped all my CDs for me on his high-end PC with multiple DVD drives.

Jim Dalrymple:

I use the “Like” system in my music services all the time because I want it to learn from my listening habits and be more personalized for my tastes. However, it seems that every service uses this system in different ways, so I talked to Apple about how you should use likes with Apple Music.

[…]

When you play a radio song, you will notice a heart—this is the like button. If you tap the heart, indicating you like that song, it does absolutely nothing to “tune” that station. Since the stations are human curated, there is no need for a tuning algorithm.

Tapping the heart does affect “For You,” the section of Apple Music that’s custom built with playlists, albums and songs tailored to your individual tastes. For You also takes into account music you add to your library and full plays you listen to. Skips aren’t really taken into account, because there are so many reasons you may skip a song—maybe you’re just not in the mood for it right now.

You can further tune the For You section. If you go to For You and there is a recommendation for an album that you just don’t like, tap and hold on the album. A menu will popup where you can choose “I Don’t Like This Suggestion,” allowing Apple Music to further learn about your musical taste.

Brian Webster:

One thing that I always wished Beats Music had (or if it did, I could never manage to find it) was the ability to see a complete list of songs that I had already “loved”.

This is now possible under Apple Music, by creating a smart playlist with a rule reading “Loved is true”. That will list all the songs that you’ve marked as loved in a single convenient playlist. Note that there doesn’t appear to be any way to create smart playlists on iOS, so you’ll need to create the playlist using iTunes on your Mac or PC, but that playlist will then automatically sync over to your iOS device (assuming you have iCloud Music Library enabled, of course).

[…]

On a related note, Beats also had a “don’t love” button (heart with an X through it) that you could press if you didn’t like a particular song very much. There doesn’t appear to be an equivalent button in Apple Music though.

Update (2015-07-20): Dieter Bohn:

I have started calling the menus spawned by 3-dot buttons in Apple Music “BlackBerry Menus” and I’m never stopping.

Apple Music: Beats 1 and AirPlay

Kirk McElhearn:

This has to be one of the more ridiculous “features” of Apple Music. A user in my iTunes forum pointed out that he couldn’t play Beats 1 Radio over AirPlay from iTunes on his Mac. I thought this must be wrong, but I tried it. The AirPlay icon is visible in the iTunes toolbar when you’re playing anything else. But when you start playing Beats 1 Radio, it disappears.

Peter Cohen:

If you’re accustomed to streaming music from your Mac to an AirPlay music source like AirPlay-equipped speakers, an Apple TV, or an AirPort Express connected to a speaker system, this might seem like a deal-breaker. But it’s not.

Hopefully the disappearing AirPlay button is just a weird bug that Apple will patch in a forthcoming iTunes update. Until then, there’s a fairly easy workaround. Just bear in mind that this sends all of your Mac’s audio to your AirPlay speaker, not just iTunes.

Paul Kafasis:

Unfortunately, when you tune in to Beats 1 in iTunes on your Mac, you’ll notice that iTunes’ built-in AirPlay sending is not offered. It’s also unavailable for other stations including NPR and ESPN radio. No explanation for this is given, though it’s likely due to licensing issues on Apple’s end. Whatever the cause, Airfoil is here to lend a hand!

[…]

Finally, we’ve had users ask us about timed recording of Beats 1. Of course, our audio recorder Audio Hijack can help you there. Unfortunately, Apple doesn’t currently provide a tune-able link to the stream, which is what a timed recording requires. Thankfully, it appears that mirrors of the stream are popping up, and you can certainly tune in with those.

Apple Music: Home Sharing and Families

Kirk McElhearn:

iTunes and iOS have a feature called Home Sharing, which lets you, for example, share an iTunes library to an Apple TV, or to another computer running iTunes on your local network. Prior to iOS 8.4, it also allowed you to share your iTunes library directly to your iOS device; when it worked.

[…]

This feature is not part of iOS 8.4; at least not for music. It’s still there for videos, in the Videos app, and, as I write this article, I’m attempting to load my video library on my iPhone.

Serenity Caldwell:

If your family members have already signed up for an Apple Music free trial, this won’t work; you’ll have to wait until after their free trial period is over to sign them up as part of your Family plan.

iPetitions:

With the introduction of iOS 8.4 and Apple Music the feature called “Home Sharing” for iTunes Music was silently removed by Apple.

Thousands of users loved to build their home music library over years to play back their music on several Apple devices, including their iPhones, iPads and Apple TV’s. Overnight Apple decided to silently remove this functionality from iOS in order to promote the newly introduced service Apple Music with monthly costs (fear remains the same will happen to Apple TV, when it gets an update to work with Apple Music in fall 2015)

Andrew Cunningham:

Last night, Apple’s Eddy Cue stated on Twitter that the company is “working to have Home Sharing in iOS 9,” confirming that the feature’s removal is only temporary. That’s of small comfort to the people who like Home Sharing and need to wait for the final release of iOS 9 in the fall before they can use it again, but it’s better than nothing.

Apple Music: iTunes Match and DRM

Kirk McElhearn:

Eddy Cue also informed a user that Apple Music will start with a match limit of 25,000 tracks, but that this will increase to 100,000 tracks with the launch of iOS 9.

It’s not clear whether this number will also be applied to iTunes Match, but one can hope that it will. Like many iTunes users, I have been clamoring for an increase in the iTunes Match limit from 25,000 tracks to something allowing me to use my music library with iTunes Match.

Kirk McElhearn:

Apple Music files cannot be copied to an iPod nano.

I guess it makes sense, that these files only play on iOS devices, but I hadn’t actually considered this before. Since the DRM on the files links to a time-limited account, you can’t copy them to a device that can’t check if your subscription is still active.

Kirk McElhearn:

The whole iTunes Match and Apple Music thing is confusing. Apple says they are “independent but complementary,” and, on first glance, they look quite similar. But when you look closely, they are very different.

[…]

When you match and download files from iCloud Music Library (without having an iTunes Match subscription), however, you get files with DRM; the same kind of files you get when you download files from Apple Music for offline listening. (These files should have DRM, so you can’t just download and keep all the music you want for $10 a month.) But if you’re using Apple Music, and not iTunes Match, Apple doesn’t make a distinction between which files were originally yours, and which you downloaded for offline listening from Apple Music.

This means that if you’ve matched your library with Apple Music and iCloud Music Library, you need to keep backups of your original files. If not, you’ll end up with files that you can’t play without an Apple Music subscription.

Serenity Caldwell:

Yes, Apple Music has a DRM component. Yes, it sucks, but it’s similar to every other streaming service. No, it does not overwrite the files on your Mac to make all your music DRM-laden.

[…]

If you use both Apple Music and iTunes Match, the Store catalog supercedes the Music streaming catalog; you should always be getting DRM-free files if you subscribe to Match. If for some reason you’re getting Apple Music files, don’t panic: it’s a bug. Try logging out and logging back in again, or deleting and redownloading the file.

[…]

So Kirk’s post is getting misinterpreted by a bunch of people—he’s a buddy, and we talked on Twitter about this with a few other folks. Here’s the deal there: He doesn’t have a hard copy of his tracks on that computer, so when he upgraded, he downloaded matched tracks and they came down as Apple Music tracks. That’s a bug, and can be fixed by deleting the track and redownloading, and worst-case logging out and logging back in again.

Nick Heer:

iTunes lacks DRM; Apple Music has DRM. That’s the difference: it’s subtle, and it’s poorly-explained. iCloud Music Library is a completely different pitch to that of iTunes Match and iCloud Photo Library, despite sounding similar, if not identical.

[…]

Apple Music and iCloud Music Library are pitched so closely, and the nuanced differences are not explained very well. Yet, these differences are incredibly important to know, because a normal person could reasonably consider their library to be safely off their computer, readily accessible when it’s needed, and largely recoverable if they were to switch to a different service.

This is an article that Serenity Caldwell should not have had to write. Not because of some of the FUDdier articles around, but because Apple should be more clear about the difference between Apple Music and iTunes. I would bet actual money that Apple wanted to — in essence — add these features onto the existing iTunes library, but were prohibited from doing so by record labels.

Serenity Caldwell:

iTunes Match isn’t the same as Apple Music. With Apple Music, for $9.99 a month, you can stream Apple’s song catalog. With iTunes Match, for $24.99 a year, you can upload up to 25,000 songs from your iTunes music library to iCloud, where you can then stream and download them—DRM-free—to up to ten other registered devices in your possession. Here’s how it works, how you can subscribe, and how you can get the most out of it!

Marco Arment (via Nick Heer):

I bet iTunes Match gets Google Readered within a year. Don’t get too attached…

Update (2015-07-14): Kirk McElhearn:

Apple has released iTunes 12.2.1, which claims to fix the problems with iTunes Match tracks being DRMed.

I installed the update, and it hasn’t fixed anything for me. Here’s an example of some albums that still show as Apple Music, and still have the incorrect artwork[…]

Nick Heer:

Let me get this straight: the bug could taint portions of a local library with DRM, and the removal procedure requires exact steps that are counter to what you may be expecting, and that following the steps you may expect instead will nuke the track almost entirely unless you use another workaround explained by Horwitz in his post? This isn’t acceptable at all.

Update (2015-07-20): Kirk McElhearn:

As you can see above, the problem was not iCloud Music Library in general, but the way it added tracks from my iPhone. I was not alone in having this problem, and, since so many other tech journalists also had these issues, I speculate that many of them are in situations similar to mine. They have multiple devices, and even, perhaps, test libraries, and iCloud Music Library did what it was supposed to do, but that behavior, designed for the “average user,” caused problems for people whose music libraries are complex.

A DJ site reported these problems, a long MacRumors forum thread discusses them, and a musician reported that Apple Music “screwed up [his] entire discography”.

Apple Music: iTunes 12.2 and iCloud Music Library

Josh Centers:

While Apple Music in iOS is a pretty good experience, it is a train wreck on the desktop. And I stop at “train wreck” only because TidBITS is a family friendly publication.

[…]

Start with iTunes 12’s labyrinth of an interface. Now, add three new tabs when Music is selected on the left: For You, New, and Connect. That’s the baseline.

From there, add in the fact that Apple Music was clearly designed as mobile first, with the desktop a distant afterthought.

Then, stir in a good dose of bugs.

Kirk McElhearn:

iCloud Music Library is a disaster. It changes artwork, alters tags, and many tracks are unavailable, having the “Waiting” iCloud status. Here’s an example of how much of a mess it is.

Kirk McElhearn:

I’ve never before recommended that you don’t upgrade to the latest version of iTunes, but I am doing so now.

[…]

I installed iTunes 12.2 on my test computer, a MacBook Pro, and quickly found that it changed a lot of my artwork and tags.

Ronald Chavez:

After I updated to the new iTunes 12.2, I was asked to enable iCloud Music Library, which is supposed to sync your tracks across your devices. Once it was enabled, iTunes randomized the majority of my roughly 25GB library. Tracks moved across albums, album art didn’t match music and artists were listed up to seven or eight times.

[…]

But it gets worse. If you’ve set up Apple Music on an iOS device, and iCloud Music Library is enabled, all of the erroneous changes are reflected in the new Music app, too.

Wil Shipley:

Unremovable, huge banner advertising YOUR station in MY iTunes makes me think @AppleMusic doesn’t care what I think.

Jeff Gamet (via Nick Heer):

iCloud Music Library was introduced with this week’s iTunes 12.2 and Apple Music release, and it’s turning out to be a big bag of hurt.

[…]

iCloud Music Library’s problems start showing up after installing iTunes 12.2 on your Mac or Windows PC. Apple’s forums are full of comments from users saying their album art, song names, and album titles get jumbled. The Who’s Quadrophenia album art, for example, could be replaced with Billy Holliday’s Lady Sings the Blues. Try to play Cream’s Sunshine of Your Love, and you might hear Vertigo from U2 instead.

Some forum posters are saying other metadata for their songs and albums has been scrambled, too, and multiple copies of songs are appearing in their libraries.

Clark Goble:

To really enjoy and make use of Music put your preconceptions behind you. Otherwise you, like me, will be constantly raving about the horrible buggy frustrating UI of iTunes.

First things first. When viewing your music switch to artist view. Yes viewing by artist is a pain when you have a nice playlist. Trust me though. If you don’t you’ll hate Music.

[…]

The biggest weakness of Music definitely is the interface. Hopefully by next year the interface is more refined. I do fear for those of us who prefer to organize our own music. It’s clear that Apple’s shifted focus away from us the past few years. Making your own playlist is just far too convoluted. However I suspect that is the traditional power user vs. typical user scenario. If one thing has become clear the last decade it is that power users are not really a concern for Apple.

Update (2015-07-14): Kirk McElhearn:

iTunes Artist View Interface Adds Content, Obscures Music

Update (2015-07-16): Jason Snell:

In any event, what I’ve discovered is that for me, Apple Music’s killer feature is that it’s completely integrated into iTunes. Not just integrated in the sense that the iTunes Store is integrated, with a separate set of pages that don’t really resemble the iTunes Library. I mean integrated in the sense that, when I find new music I like, I can click the plus icon and add it to my iTunes Library.

I don’t know what I was expecting from Apple Music integration. I guess I assumed that when I added a track to “my library” from Apple Music, it would go to some special Apple Music tab, or playlist, or library. Nope—that music just shows up in the My Music section of iTunes, mixed in with all of the stuff I’ve bought over the years.

[…]

I can also see just how insidious this approach is. My music library is no longer pristine, no longer a collection owned by me. Now I’m acquiring albums and tracks not by buying them, but by clicking that Add to Library button.

Update (2015-07-20): Doug Adams:

The current track and current playlist properties return a -1731 “unknown object type” error when run against a currently playing Apple Music track. That’s going to be a problem for scripts and apps that use those properties to identify a playing track. I couldn’t say if this is intentional or a bug.

The iTunes XML reports the main Library name as “####!####”. This is not necessarily AppleScript related, but several of my scripts may refer to this “Master” playlist by obtaining its name from the XML.

Apple Music: General

Samuel Hulick made a great slideshow called “How Apple Music Onboards New Users” (via Steven Woolgar).

Serenity Caldwell:

Here’s what Apple Music is, what it’s not, how it compares to other services, and what you’ll be able to find on your iPhone, iPad, Mac, or PC.

Nick Heer:

It’s only been a day since Apple launched their newest streaming music service, so the thoughts I have about it are fairly preliminary and would probably comprise several shorter posts. For convenience, they’re here in a bulleted list.

Eli Schiff:

The iOS Music icon again evolves—or devolves? Originally the iPhone music app was called iPod, but that soon went out of fashion and Apple opted for the title “Music,” which remains today. The most impressive shift of course was between iOS 7 and iOS 8 in which Apple designers went to the trouble of reversing the gradient.

David Pogue:

In Apple’s glory years, Steve Jobs turned simplicity into an art form.

“Being focused means saying no to the hundred other good ideas that are out there,” he once said. “You have to pick carefully. Innovation is saying no to 1,000 things.”

It’s starting to seem as though Apple no longer abides by that religion. The first two major post-Jobs initiatives from Apple are powerful and important, but they’re also bogged down by too many features and a confusing design. First, there was the Apple Watch — and today, there is Apple Music.

Cezary Wojcik (via Mike Rundle):

Unfortunately, I don’t see myself switching from Spotify anytime soon. I really tried to convince myself to give it a chance, but Apple Music is just simply awful to use. Everything I would want to do would either take forever or is simply not possible.

[…]

So, every song I had to delete required 4 taps, most of which had to wait on some kind of animation to finish. Just when I thought that it couldn’t get worse than deleting the songs individually from the playlist, I found something literally twice as bad (4 taps vs 2 taps).

Justin Blanton:

Want to turn off Apple Music Connect and replace its tab with a Playlists tab?

Joe Rosensteel:

I started poking around with Apple Music, and I’ve been particularly interested in how it has functioned offline in comparison to its predecessor. Like, 1 in 5 buttons in the main interface show you a white screen with gray text saying that you are, in fact, offline. It’s a barrel of fun, turn on Airplane Mode and give it a whirl.

Fortunately, you can make tracks, and albums, available for offline listening, but there’s no genius playlist functionality. Finding it in an ellipsis menu (not all ellipsis menus offer the function) yields a modal dialog that you need to be connected to cellular or WiFi to create a genius playlist.

This was not a problem in the previous iteration of the app, because the genius data was updated when you synced your phone, and available offline.

Collin Allen:

Was going to comment on how well Music’s color sampling works, then got this. Thought UI was disabled.

Nick Bradbury:

The fact that Apple usually does a phenomenal job with UX makes things like this such a disappointment.

Dan Moren:

My favorite tidbit, though, is from Mossberg’s piece:

Siri was able to effectively respond to commands like, “Play the top hits from 2007″ or “After this song, play ‘Heartbreak Hotel.’”

I have wanted the ability to easily queue up the next song on my iPod and iPhone since, oh, 2001.

Kirk McElhearn:

You can’t stream everything that in the iTunes Store on Apple Music. There are lots of labels, and artists, who aren’t playing the streaming game. Notable labels that are missing are ECM, the jazz label, and Hyperion Records, the classical label, both important independent labels in their genres.

But there are also individual tracks, or parts of albums, that are unavailable. I came across a few of them yesterday.

Kirk McElhearn:

For starters, many parts of Apple Music are not designed for classical music. The whole playlist aspect of the service is clearly not ideal for this type of music, which doesn’t contain “songs,” but rather works, often of multiple movements. So the For You section of Apple Music, which offers playlists and albums to check out, won’t be of much help.

[…]

Apple Music fails as far as presenting metadata about classical music. Looking at a number of albums in the New section, I find that many of them don’t display the names of the composers whose works they feature.

Kirk McElhearn:

I know Apple Music is just getting started, but they can certainly do better than just provide “Classical Music for Elevators.” Maybe Apple needs to hire some classical music “curators.”

Kirk McElhearn:

When you visit the Music app on the Apple TV, there is still a tab for iTunes Match, which is all but invisible in iTunes and on iOS. But nothing about Apple Music. It’s as though Apple forgot about the Apple TV.

[…]

It’s clear that video is going to be a part of Apple Music – you can already view music videos, if you can find them – but will Apple try and make a new MTV? I don’t buy the idea of live streams of DJs, but they do already have the iTunes Festival, and could certainly add more.

Adam Jackson:

My only issue that remains with streaming music…It requires a data connection.

[…]

The thing is, I don’t see any reviewers mentioning this. I guess everyone but me has 4G / LTE everywhere?

Update (2015-07-14): Kirk McElhearn:

Apple Music Doesn’t Display Your Listening History and This Is a Big Mistake

Safari Content Blocker and Web Advertising

Dean Murphy:

Ok, so the website I decided to try “fixing”, is one that I see linked often, iMore. Let me start this by saying I really like the content that iMore provide and enjoy the personas of their staff on their many podcasts (Debug is one of my favourites!), but I hate the experience of their mobile website as it has several ad’s by many different providers, all tracking me across all different sites. They have super tiny ‘close’ buttons that are near impossible to hit, they follow you as you scroll and there is a slow loading full page interstitial that loads on every page refresh. Urgh!

[…]

With no content blocked, there are 38 3rd party scripts (scripts not hosted on the host domain) running when the homepage is opened, which takes a total of 11 seconds. Some of these scripts are hosted by companies I know, Google, Amazon, Twitter and lots from companies I don’t know. Most of which I assume are used to display adverts or track my activity, as the network activity was still active after a minute of leaving the page dormant. I decided to turn them all off all 3rd party scripts and see what would happen.

After turning off all 3rd party scripts, the homepage took 2 seconds to load, down from 11 seconds. Also, the network activity stopped as soon as the page loaded so it should be less strain on the battery.

Rene Ritchie:

I don’t block ads because, as someone who works for a site that has ads, I understand the cost of content and the current realities involved in paying for it at scale. (I don’t skip podcast sponsorships—the ultimate in native, intercept ads—for the same reason).

Both via Nick Heer:

His “response” article — which, I should point out, is entirely text-based, unlike a media-heavy review — weighs in at a whopping 14 MB with 330 requests.

John Gruber:

Rene Ritchie’s response acknowledges the problem, but a web page like that — Rene’s 537-word all-text response — should not weigh 14 MB.

[…]

With Safari Content Blockers, Apple is poised to allow users to fight back. Apple has zeroed in on what we need: not a way to block ads per se, but a way to block obnoxious JavaScript code. A reckoning is coming.

Rene Ritchie (via John Gruber):

First, the content size issue. 14MB is infuriating. My guess is that he was getting a video ad on the page that’s no longer being served. We’ve been testing internally and getting consistently under 4MB for that page, which is still hefty.

[…]

While we sell premium ads directly to advertisers, that only fills a small subset of the required “inventory” to support the network. Some 85% of ads we served last month were “programmatic”—provided by ad exchanges like Google Adx and Appnexus. Those exchanges are pretty much black boxes. We get a tag, we insert it, and ads appear.

[…]

Each ad gets its own iframe, so load is asynchronous and, if one fails, it doesn’t kill the entire site. Unfortunately, that also means each one fires its own trackers, even if those trackers are identical across ads. It’s terribly inefficient.

[…]

We also have no ability to screen ad exchange ads ahead of time; we get what they give us. We can and have set policies, for example, to disallow autoplay video or audio ads. But we get them anyway, even from Google. Whether advertisers make mistakes or try to sneak around the restrictions and don’t get caught, we can’t tell. It happens, though, all the time.

Glenn Fleishman (via John Gruber):

Advertising, analytics, social media, and other tracking networks use JavaScript, tiny images, and other embedded methods to install tracking IDs on your browser when you visit sites that incorporate their signals. This might be a site that uses Google Analytics, Doubleclick, GeoTrust, or dozens of others—or even dozens on one site.

All legitimate networks offer some kind of opt-out method, but many work poorly, and you have to opt out often for every browser by network, and sometimes only for a limited period of time. And, as with the Do Not Track quibble, opting out of tracking can mean you’re tracked with a promise to not use identifying information.

Because of all this, users have increasingly installed ad-blocking software, which throws the baby out with the filthy bathwater. Poor baby! The baby is the revenue from advertising that allows sites such as Macworld and hundreds of thousands—or maybe millions—of others to pay the bills that make publications go from a part-time self-employed blogging gig to a newsroom of hundreds of reporters.

[…]

I like [Firefox] Tracking Protection because of its integration and seemingly light hand in what it does. But Disconnect (which helped provided the blocklist for the feature), Ghostery, and others offer similar or better features. Ghostery, for instance, shows you a count of how many tracking elements on a site when the page loads, and lets you block whichever you like. Ghostery is focused on privacy, not malice. Disconnect has its feet on both pedestals.

Previously: Introduction to WebKit Content Blockers.

Update (2015-07-12): Nick Heer:

The rise of content and ad blockers has required companies to get creative about how they show us ads. Buzzfeed has mastered the art of “native” advertising on the web, but that also kinda sucks for readers because it feels deceptive. The short sponsor posts popular among many sites feel more honest, but they’re straddling a fine line between a clearly-marked sponsor post and a native ad.

It’s a hard question: how do you get paid on the internet in a way that feels respectful to readers? Is it as simple as clearly labelling sponsored content as such? Is there a better way?

Update (2015-07-22): TJ VanToll (via Chris Johnson):

This article makes for a good showcase of web cruft. All I wanted to do was read about psycopaths, as one does, but before reading I had to sift through a bunch of junk that I don’t care about—like social buttons, the temperature, and a terms-of-service modal — all for an article that’s about 2,000 words. I can’t even see the start of the article on my oversized iPhone 6+.

Loading this article took 200+ HTTP requests and used ~2MB of data. The article took about 3 seconds to load on my WiFi, and web page test says it would take about 13 seconds to load on an average mobile network.

I don’t bring up this example to single out CNN, because, as sad as this is to say, this article is now representative of the average web experience. According to the http archive, the average web page surpassed 2MB this May, and is now at 2.08MB. It’s not hard to find a far worse example out there.

Update (2015-07-29): See also this episode of The Talk Show with Jason Snell.

System Integrity Protection (a.k.a. Rootless)

WWDC 2015 Session 706 (video, PDF, tweets):

And for the same reason that you shouldn’t put all of your eggs in the same basket, you shouldn’t rely on a single layer of protection to defend the device, because no matter how bulletproof, or water resistant, or shock absorbent this layer is, when it starts failing you, then it’s complete game over.

Instead, you should rely on multiple layers of protection, ideally with different security properties that will delay the advance of an attacker and reduce your attack surface.

[…]

So the reality is that once you have code running on the Mac, it’s actually not that hard to become root, and once you are root, you have full control of the machine.

Which means that any piece of malware is actually one password, or one vulnerability away from taking full control of the device.

[…]

We need a layer that will eliminate the power of root on the machine and protect the system by default, as it was installed by Apple on the machine.

[…]

This is what System Integrated Protection is.

It is a new security policy that applies to every single process running on the system.

[…]

If you install anything in /bin, or /sbin, or anywhere under /usr like /usr/bin, /usr/lib, /usr/libexec, then you need to move this content into the appropriate subfolder of the /usr/local folder, because that’s the only location that is now available to third-parties.

[…]

For one, the task-for-pid and the processor-set-tasks SPI will now fail if they are called on a restricted process.

And will set an 0 to EPERM.

Which means that if part of your product relies on being able to attach to a system process at runtime, for instance, the Finder, and that you expect to be able to inject code into the Finder, that is not going to work anymore.

[…]

And finally, if you use dtrace, all dtrace probes that target a restricted process will not be matched anymore, which means you won’t be able to see an interaction between the process and the kernel.

[…]

If you try to invoke lldb even as root and try to attach to the Finder, then this is going to fail.

[…]

Now, because root can actually set a NVRAM setting and we can’t trust root to do the right thing here, it means we cannot have the configuration mechanism in the OS itself.

If you want to change the configuration, you need to reboot your machine in Recovery OS, and you can do so by holding the Command+R key on boot.

Then all you have to do is launch the Security Configuration application from the Utilities menu, and check the System Integrity Protection box, apply and reboot.

Daniel Jalkut:

It’s awful restrictive. I ended up needing to disable it to even to attach to Dock with lldb.

Landon Fuller:

Add Dropbox – which used Finder code injection – to the list of things that’d be impossible to ship on today’s OS X.

Landon Fuller:

Mac OS’ original support for running multiple applications at once started life as a 3rd-party Finder extension.

Karsten Weiss:

However, one of DTrace’s selling points is/was that you can use it anytime on a production system (without reboot).

Dave Nanian:

In our investigation, we’ve found that a new Extended Attribute -- com.apple.rootless -- is used to mark files and folders with this new protection. No process other than certain Apple-signed-and-authored ones can remove or write this attribute, and files and folders marked with this attribute cannot be changed.

[…]

Since we can’t write the com.apple.rootless EA, SuperDuper removes it during the copy. That means the backup -- while fully functional and bootable -- is not an “exact copy” of the source. Specifically, SuperDuper! must disable the system protection feature on the backup, and cannot recreate it when you restore.

[…]

It’s easy to regain full system protection features: you simply need to reinstall the OS from the App Store. You can do this at your leisure, but doing it as soon as possible means you’re less vulnerable (even though that vulnerability is quite small). It’s a painless process, and it writes the fresh OS under your existing applications and data. As an added benefit, it will speed up your boot process, since it’ll recreate certain caches that non-special-Apple-programs can no longer update.

Rootless also affects DropDMG, since it installed its command-line tool in a folder that is now off-limits. I’ve released a public beta that adds compatibility with Mac OS X 10.11.

Gwynne Raskind:

Getting compilers and associated tools to be independent of the traditional UNIX path structure is brutally hard.

Rosyna Keller:

rootless is likely going to cause issues with badly misbehaving Linux/POSIX software.

Jonathan Wight:

also seen issues with some python modules (lxml) too. Ended up filing bugs all over the place and told “not our problem”

I’m worried about compatibility with LaTeX, as well as the long tail of Unix tools, which will probably not be adapted for a long time, if ever.

Update (2015-07-12): Gwynne Raskind:

Also notice that the EA mentioned by Dave Nanian is, at least conceptually, just a reimplementation of chflags(SF_IMMUTABLE);

chflags() can only unset SF_* in single-user mode, whereas the EA appears to be gated by entitlements. The result is the same.

Landon Fuller:

Real difference is unrelated to security: EA is tied to Apple’s code signing approval rather than local admin control.

Gwynne Raskind:

That’s what inspired a new impl in the first place: Local admin control is assumed to be too stupid to be safe.

Gwynne Raskind:

I actually think it’d be much harder to social-engineer users into turning off SF_IMMUTABLE than into booting Recovery

Gwynne Raskind:

Use of the existing options for this kind of control would keep them from being able to bypass it for themselves.

Landon Fuller:

Yes, and keeping it Apple-only makes it easy to flip a switch and make it mandatory, like iOS.

I have to assume that’s the end-game. They’re whitelisting/grandfathering in user reqs that would cause blow-back while at the same time making it impossible for new entrants to introduce new requirements. Hard to see any other end-game

Update (2015-07-15): Dave Nanian:

Apple fixed the problem with copying the “com.apple.rootless” attribute in the Public Beta! So, with the release of our Beta 2 (download below), we’ve included the ability to copy with that EA preserved, and thus system protection is maintained on the copy as well. Plus, there’s no need to erase when restoring.

This is all great news for users: basically, copying will work as it always has.

Landon Fuller:

The “security” of blocking code injection vs. the loss of productivity in only being able to use solutions Apple invents.

BinaryAge (tweet):

Both TotalFinder and TotalSpaces2 work by injecting code into processes that are part of OSX. They change the way those processes work, but they don’t change the underlying system - they just add features whilst they are running. If you quit TotalFinder or TotalSpaces2, those processes restart and system returns to its original state.

However, in El Capitan OSX 10.11, this kind of modification will be disallowed by a new feature called “System Integrity Protection”. It is also known as “Rootless”. The feature prevents both modifications to your system files, and to system processes whilst they are running (even if you enter your password for administrator access).

So in a normally configured Mac, TotalFinder and TotalSpaces2 cannot run.

Frederic Jacobs:

What Apple is doing with this new OS X is the same thing they’ve been doing on iOS, protecting the boot chain by signing the whole boot process. This prevents (in theory) an attacker from hijacking the boot process to inject persistent malware. But unfortunately, this makes it really difficult to monitor your own machine against compromise. The only forensic analysis you can apply on such a system are black box analysis techniques since you can’t have any insights about what’s going on outside of user space. Malware becoming incredibly hard to track down if it used an exploit to enter kernel space. The hope is that it would not be able to find a persistence mechanism given that the boot chain is signed.

[…]

For now you can still disable most of these protections as a user/developer, but most of your users won’t (and it’s probably safer for them not to). As on iOS, the Mac’s distribution channels might be entirely controlled by Apple some day. It will be increasingly difficult to provide any feature that is not blessed by Apple. Something like randomizing your MAC address or verifying that your filesystem is encrypted like it should be might become impossible. How much of a “general-purpose computer” does it become if Apple acts as a gatekeeper to what operations can be ran on it?

Update (2015-07-16): Will Robertson:

The MacTeX people are already on top of the changes in El Capitan -- TeX Live will in the future be located inside /Library

Update (2015-07-29): Craig Hockenberry:

The OS X “rootless” mode has benefits, but it’s going to make things very tough for the next generation of designers.

Update (2015-08-13): See also Accidental Tech Podcast #128.

Mac OS X 10.10.4 and Apple TV

Marco Arment:

It’s been almost two weeks since my Apple TV refused to see my iMac’s iTunes share, or since I had to restart iTunes and reboot the Apple TV or disable and re-enable Home Sharing to get them to (maybe) see each other.

In fact, it’s been almost two weeks since I’ve seen nearly any of the annoying, recurring problems that made me write, back in January, that Apple’s software quality had lost the functional high ground.

Not coincidentally, it’s been almost two weeks since Mac OS X 10.10.4 was released, which replaced the problematic discoveryd with the older, more reliable mDNSResponder. These system processes are responsible for tracking the network’s computers, names, and shared services, and discoveryd’s unreliability in these tasks caused erratic network problems like mine for a huge portion of Apple’s customers.

Currently, neither of our Apple TVs can stream video from the iTunes Store, either via Ethernet or Wi-Fi. Restarting the Apple TV no longer helps. The only workaround seems to be to use AirPlay from an iOS device. So hopefully there is another update forthcoming.

Saturday, July 11, 2015

CSS Font Family for San Francisco

Craig Hockenberry:

As a part of this abstraction, there’s now a new generic family name: -apple-system-font. The markup looks like this:

body {
  font-family: -apple-system-font, HelveticaNeue, LucidaGrande;
}

Unfortunately, it’s very poorly documented. Most of the information I’ve found about this name comes from the WebKit source code. It certainly feels like work-in-progress.

Update (2015-07-12): Nick Heer:

What’s kind of fun about this discovery is how -apple-system-font — the self-descriptive system font declaration — renders on the current OS X/iOS releases. If you open his test page, you can see some subtle differences between Helvetica Neue (the red text) and the system version of Helvetica (the last three examples).

Update (2015-07-27): Myles Maxfield:

Web content is sometimes designed to fit in with the overall aesthetic of the underlying platform which it is being rendered on. One of the ways to achieve this is by using the platform’s system font, which is possible on iOS and OS X by using the “-apple-system” CSS value for the “font-family” CSS property. On iOS 9 and OS X 10.11, doing this allows you to use Apple’s new system font, San Francisco. Using “-apple-system” also correctly interacts with the font-weight CSS property to choose the correct font on Apple’s latest operating systems.

Previously: San Francisco as the Mac System Font.

Tuesday, July 7, 2015

ComiXology 3.7 Makes the Best of the Amazon/Apple Situation

Jason Snell:

When ComiXology was bought by Amazon last year, the company almost immediately ripped all in-app purchases out of the iOS version of its popular comic-book-reading app. Amazon wasn’t interested in paying Apple 30 percent of its in-app sales, Apple wasn’t interested in negotiating terms, and comic book readers could no longer buy and read comics with a couple of taps.

[…]

The lack of in-app purchases is a dealbreaker for some people, though I don’t mind it too much—especially now that the ComiXology app has been redesigned, and thanks to some major improvements to the ComiXology web site. Recently ComiXology added one-click purchases to its site, which was already redesigned to work great on the iPad.

[…]

Is this approach as good as buying issues via an in-app purchase? No, it’s not. And without in-app purchases, you lose the ability to finish an issue and tap to buy and read the next issue in the series—a dangerously expensive form of binge reading. But ComiXology has clearly put a lot of thought into making the experience as good as possible, given Amazon’s refusal to cut Apple in on sales.

A Complete History of Unix, in One GitHub Repo

Unix History Repository (via Mike Hoye):

The goal of this project is to create a git repository representing the Unix source code history, starting from the 1970s and ending in the modern time. To fulfill this goal the project brings data from early snapshots, repositories, and primary research. The project aims to put in the repository as much metadata as possible, allowing the automated analysis of Unix history.

[…]

The project has achieved its first major goal with the establishment of a continuous timeline from 1972 to 2015. The repository contains

  • snapshots of V1, V3, V4, V5, V6, and V7 Research Edition,
  • Unix/32V,
  • all available BSD releases,
  • the CSRG SCCS history,
  • two releases of 386BSD,
  • the FreeBSD 1.0 to 1.1.5 CVS history,
  • an import of the FreeBSD repository starting from its initial imports that led to FreeBSD 2.0, and
  • the current FreeBSD repository.

Dash 3

Kapeli:

Dash 3 for OS X is the biggest update I’ve ever worked on and I’m happy to announce it’s finally available. It comes with a lot of features that you, the users, have requested and some features which I’ve always wanted to add but never had the time until now. And lots of bug fixes, of course.

[…]

Dash will now use a lot less disk space by keeping docsets archived. Files will only be unarchived on-the-fly when they’re needed. Docsets take up to 10x less disk space while archived.

[…]

Use Annotations to extend any documentation page by adding comments, examples, warnings or bug reports. Annotations are stored online and can be public or private. You can also set up teams to share annotations with your coworkers or friends.

The Annotations Server is open-source so you can set up your own server if you want, for ultimate privacy.

The annotations are a great idea.

I can’t believe that Apple is still allowing this free trial with an In-App Purchase to disable the nag screen.

Update (2015-07-07): Kapeli:

I never had an issue getting accepted due to “nag-to-purchase”. They don’t see it as trial, maybe same thing as allowing lite apps.

As far as I can tell, their definition of trial is time-based only.

So perhaps Apple sees nagware as more along the lines of “pay to remove the ads,” which lots of apps do. This seems like a great fit for an app like Dash, where making the full functionality free is not a problem because a big part of the app’s value is how fast it is.

VoodooPad Cryptography Overview

Plausible Labs:

This document provides a high-level overview of the encryption implementation released in VoodooPad 5.1.4 (Mac) and VoodooPad 5.1.4 (iOS).

Additional technical details are available via the VoodooPad Crypto Specification.

It’s good to see that they fixed the problems with the old implementation and are being open to hopefully find any problems with the new one.

Sunday, July 5, 2015

GPS in Airplane Mode

Dave Rullo:

Airplane Mode turns off all internet functions, phone calls, and texts. Prior to iOS 8.3, Airplane Mode also disabled the GPS in your iPhone/iPad, which crippled most of Gaia GPS. Now, with the latest iOS updates, you can use Airplane Mode, and Gaia GPS will be able to plot you on the map, and record your trip.

Presumably the GPS does not perform as well without access to cellular towers or Wi-Fi. But this could potentially save a lot of battery life when you’re in an area with poor cell service. Plus, it would let you track your flight process while on an airplane, if you had the maps preloaded.

Installing Fonts on iOS

Norbert Lindenberg:

iOS comes with a selection of fonts that cover the major writing systems of the world. Some apps, however, need to install additional fonts for system-wide use. […] This article describes how such apps can package and install fonts, based on my experience bundling the Ubud font with the Balinese Font and Keyboard app. Note that this is about system-wide use – if you need to bundle a font just for use within your own app, Chris Ching has a tutorial for that.

[…]

For some parts of the Unicode character set WebKit now looks for fallback fonts, for other parts it doesn’t (Balinese was one of the lucky scripts). Recent code changes in WebKit indicate that this will finally be completely fixed in iOS 9.

[…]

While support for third-party keyboards was highlighted as a new feature in iOS 8, support for third-party fonts received much less attention. This is probably because the support that exists was primarily targeted at enterprise customers: Fonts for use across apps are packaged in configuration profiles, which otherwise serve to configure virtual private networks, disable games and unsafe web sites, locate network printers, and do other things that matter in corporate environments.

[…]

[Apple] Configurator creates and uses a self-signed certificate, which iOS doesn’t trust, so you get the same number of warnings, just saying “Not Verified” instead of “Not Signed”. […] I ended up buying a COMODO code signing certificate from KSoftware, which cost US$95 and quite some time because their support for Macs is a bit flaky.

[…]

Installed fonts do not get updated automatically. When the user installs a new version of your app that includes a new version of the font, you likely have to remind the user to re-install the font.

So I guess Font/DA Mover is no longer the hard way to install fonts.

Another Downside of Automatic App Updating

I have long had Settings ‣ iTunes & App Store ‣ Automatic Downloads ‣ Updates enabled but Use Cellular Data disabled. I thought the latter would protect me from unreliable cellular connections and wasting bandwidth. Unfortunately, I did not account for unreliable hotel Wi-Fi or what I consider to be an OS bug.

On a recent trip, iOS started downloading an update to Editorial but apparently couldn’t finish the process. This would be OK except that it left the app in an incomplete state where the home screen showed that it was in the process of updating, but the app would not launch. So I was unable to access my important notes. (Other Dropbox text editors would have worked but didn’t have the files cached on the phone.)

Previously: Downside of Automatic App Updating.

Friday, July 3, 2015

Using Siri

My experience with Siri keeps changing. I’m pleased with the recent change where you no longer have to confirm before a reminder is added. Instead, you just tap Remove if Siri got it wrong. There’s still no good way to edit if Siri got it partially right, though. And sometimes Siri is repeatedly so far off that it’s not even enough to jog my memory; I wish I had just stopped to type.

It is maddening how many times I will see that the phone has cell service and dictate my reminder, only to have Siri spin seemingly forever or complain that it doesn’t have an Internet connection. Couldn’t it have told me that sooner? Or recorded what I’d said so that it could be replayed when a connection becomes available? Since Siri requires a connection, it shouldn’t even play the “start talking” prompt unless it knows that it can reach the server.

Arq Supports Amazon Cloud Drive Backup

Stefan Reitshamer:

You can now store your encrypted Arq backups in your own Amazon Cloud Drive account!

Amazon Cloud Drive includes unlimited storage for $59.99/year, so you can back up everything. Amazon provides a web interface to upload and download files, but Arq uses its API to upload directly to the cloud.

Previously: Amazon Cloud Drive.

Twitterrific’s New Facial Detection

Ged Maheux:

By far the coolest of these improvements is the use of Apple’s facial recognition APIs to improve image previews. What does that mean exactly? It means that as Twitterrific displays media thumbnails in the timeline (pictures, videos, etc), the app tries to detect faces and frame the thumbnail so faces are always showing. In short, if Twitterrific sees a face in a tweet, it tries to make sure you see it too!

Should I Use a Swift Struct or a Class?

Drew Crawford:

A lobby that wants you to believe that one weird trick will make your codebase better, replacing the cargo-cult Agile practices with equivalently useless cargo-cult functionalist practices.

[…]

As we’ve established, most types (say, 90%) can implement funcs in a non-mutating way. But to quote another Zawinski law, “Everybody uses a different 10%”. […] Similarly, for every type that is sufficiently large, it conforms to some non-mutating protocol with a mutating implementation. […] Now. How are we going to solve this problem? Well, the mutating inner class.

[…]

It is not wrong to wrap a class in a struct. This is how Swift.Array works under the hood. […] The difference is, A) they have been really clever about hiding the inner class, which involves a lot of code and neat tricks that you can learn more about in the WWDC session (e.g., this is an advanced topic, that you should not sprinkle liberally throughout your codebase where the muggles will see it) and B) there is nothing fundamentally classful about arrays. Arrays are, from a certain point of view, a kind of value. So it makes sense for them to be a value type.

[…]

The moral of this story is that immutable, pure functional programming does not always work even in the literal textbook examples. A mutating implementation of the Sieve is both simpler and faster, and far from helping, functional programming has actually hindered the exercise of writing fast, maintainable code.

[…]

Finally, the official Apple guidance concludes with this dire warning:

In all other cases, define a class, and create instances of that class to be managed and passed by reference. In practice, this means that most custom data constructs should be classes, not structures.

Update (2015-07-05): David Owens II:

The fundamental problem I find with the piece is that it creates a false selection of choices to the problem statement posed while misrepresenting much of the advice about choosing when to use value types. I think it also confuses the purpose of the proposed “inner class device”.

[…]

The entire purpose of the “inner class” is to provide value semantics while maintaining efficiency of implementation. In addition to that, if some of the quoted advice in the article had been followed with regards to struct usage, a different conclusion would have presented itself.

Drew Crawford:

I agree with you that inner mutability “feels” wrong in the CanSend example. But the suggestion that inner mutability should never ever be used is wrong, because of Swift.Array. So we need some rule to distinguish when inner-mutability-with-structs is bad from when it is okay. If the rule was “never do any mutation inside a struct” then we would not have value-typed-arrays to begin with.

(In fact, we didn’t have value-typed-arrays to begin with. I have a serious theory that it was exactly the rigid application of “never have inner mutation” inside Swift that initially led to the silly behavior. Recall that in early betas, the value/reference behavior of arrays used to hinge on whether or not the operation caused a resize--which is exactly those cases in which the array implementation needed to mutate the free block chain.)

David Owens II:

I think it’s safe to say that using inner classes to simply break the mutability contract of a protocol is a poor reason that’s going to lead to all sorts of bad behavior (like the original arrays in Swift).

[…]

I’m saying that I think that line is somewhere near the area of, “I really want to present a value-semantic type, but the performance is going to suck if I do, so let’s create an implementation with a backing inner class”.

Update (2015-07-22): Mike Ash:

The answer is actually really simple: use structs when you need value semantics, and use classes when you need reference semantics. That’s it!

[…]

Thus the fundamental question to ask when deciding which one to use is: does it make sense to copy this type? Is copying an operation you want to make easy, and use often?

Copyright © 2000–2024 Michael Tsai.