iCloud Drive Breaks the macOS Command Line
Apple’s current engineering solution breaks consistency of file names and paths. When a file has been evicted from local storage, and only exists in full in iCloud storage, the local stub file uses the previous name prefixed with a stop/period, and gains the extension of .icloud. When that file is downloaded to local storage – something which can be triggered by all sorts of events – the leading stop/period and the extension are stripped.
[…]
Many commands and scripts can safely ignore files which the user has placed in their iCloud Drive. But the moment that a user enables Desktop & Document Folders to be stored in iCloud, with Optimize Mac Storage enabled, file names in ~/Documents are affected, and commands and scripts will fail when run on one of the most important and active directories on most macOS systems.
[…]
Apple’s own most robust tool for locating files, the Finder alias, is broken by iCloud.
[…]
Inevitably, all hard and symbolic links made to evicted files are also broken by their eviction.
Faced with the problems posed by iCloud, a lot of commands, shell scripts and other scripting becomes inordinately complex, and in some cases impossible. Apple needs to continue to evolve the iCloud interface, making it consistent with the fundamental needs of commands and shell scripts. If it doesn’t, but continues to converge with iOS, it will undermine macOS itself.
Update (2018-05-10): See also: MacInTouch.
5 Comments RSS · Twitter
I don't think this has anything to do with iOS. It's just poorly engineered solution that worked while it was self-contained and got expanded to rest of the OS without thinking. They simply need to give a little more attention to "power" users. iCloud synced keychain has similar problem - it can't be accessed by any tools that can work with normal keychain, including Apple's own `security` command line tool, because it's special.
They need to make is so it doesn't use weird paths, weird file names, and weird behaviors. A normal folder, with normal filenames, that just happens to get synced to Apple Cloud. Dropbox does it so...it's possible.
@Ed: Does Dropbox replace the local file with a 'symlink' to the remote one? (I'm guessing not.) That's the challenge.
Tend to agree with @Mayo's assessment: the extant implementation is a fast-n-filthy hack that was done for iOS, where apps access the file system via high-level Cocoa APIs, not low-level POSIX. Fine for a quick proof-of-concept; survivable for a not-really-POSIX-platform like iOS; a flaming bag of cat poop for macOS.
Renaming files to indicate local-vs-remote presence is pretty horrific; either Apple doesn't understand the (misleadingly named!) Unix filesystem, or the Unix filesystem doesn't understand Apple. The Unix FS is an abstraction; a tree-shaped namespace where the only requirement is that each node in that tree support read() and write() operations—not a literal filesystem. There's absolutely no obligation for nodes in that tree to map to literal files on a literal disk drive.
The right solution would be to replace 'ordinary file' objects with 'special file' objects that handle remote read/write and local caching transparently, decoupling backend storage from frontend presentation. The first question is whether the Unix FS is flexible enough to support that particular abstraction. Special files in the Unix FS are largely limited to representing I/O devices (Plan 9's FS was much more advanced and could represent pretty much anything; a shame it failed to supplant Unix).
The second question would be whether Apple has engineers with both the depth of Unix knowledge AND the authority to make such an implementation happen, and the third question would be whether Apple's ADD marketing-driven development schedules would allow them the space to do it right. Personally I'd reckon the odds of any one of those at 50%, giving us an 87.5% likelihood of Apple ballsing the client-side part somehow. And considering Apple STILL doesn't get networks and IPC after 40 years in the game, I still wouldn't have huge faith in them not buggering up the server side as well.
Frustrating for Macs users who expect Everything to Just Work, but this is why Alphabet (Google) is starting to eat their consumer/educational markets: unlike Apple they were born with network programming in their blood. Even technical-debt-saddled Microsoft have woken up and are getting their act together, yet Apple appears to be asleep at the wheel. Either they'll figure out where the hell their platforms and integration needs to go and pull their pants up pronto, or there'll most likely be a mass exodus to Chromebooks (consumers) and Windows (professionals) in the next decade as they fall further and further behind. Given the current Apple leadership, I wouldn't be putting money on it either. All of this has happened before; all of this will happen again.
