Archive for July 31, 2023

Monday, July 31, 2023

iCloud Drive Troubleshooting, Metadata, and Custom Icons

Howard Oakley:

Fixing problems with iCloud and iCloud Drive is completely different from any other troubleshooting in macOS, because:

  • Almost all problems are delays in or failure of synchronisation.
  • Documentation is almost entirely lacking.
  • There are almost no tools to help diagnose or fix problems.
  • Even those experienced in reading the log find it hard to use for diagnosis of iCloud problems.
  • There’s a very limited range of potential solutions.

The main problem I have with iCloud Drive is that (even small) files added from my iPhone often take 8 or more hours to be uploaded, even with the phone on Wi-Fi and a small total amount of data stored in iCloud Drive. There are even fewer troubleshooting levers to try in iOS than in macOS. Sometimes restarting the phone helps; other times I just need to wait or share the file in a different way if I need it sooner.

Howard Oakley:

The illusion that all xattrs are retained by iCloud Drive when read from the same Mac that uploaded them isn’t the result of those xattrs being stored alongside the file’s data in iCloud Drive. Instead, those xattrs are accessed from the local file system. Eviction thus appears to change local metadata by deleting large xattrs, and all xattrs.


For example, although I’m sure they have long since been deprecated, Resource Forks can still be used to provide a custom thumbnail that’s displayed in preference to the default set by QuickLook. As that is accessed from the local file system on a Mac that uploaded the file to iCloud Drive, on that Mac the custom thumbnail will be displayed until the file is evicted; when it’s downloaded again, the Resource Fork will have been stripped, and the custom thumbnail has vanished. But on a different Mac accessing the same file in iCloud Drive, the custom thumbnail won’t have been shown at all.

Although that might seem an edge case, vanishing metadata can have other significant impacts on apps that rely on it.


Flags can be upper or lower case letters C, N, P and S, and invariably follow the # separator, which is presumably otherwise forbidden from use in a xattr’s name. Upper case sets (enables) that property, while lower case clears (disables) that property.

Howard Oakley:

Even if you were able to upload the Icon? file complete with its xattr, there’s another problem to overcome. In APFS, your local file system, directories can have xattrs attached to them, and often do. iCloud Drive doesn’t appear to have any equivalent that can be associated with xattrs, so there’s nowhere to store the xattr containing the flag to indicate that folder has a custom icon.


This xattr stripping also comes into play for custom file icons that are evicted, then downloaded and viewed on the same Mac that added the icon. As long as a file isn’t evicted, it still uses the xattrs stored in its local file system metadata, including, but when it has to be downloaded from iCloud Drive, that xattr is stripped, so removing the custom icon. This explains why custom file icons can mysteriously vanish, even when they’re viewed from the same Mac that created them.


Since writing my previous account of the behaviour of xattrs in iCloud Drive, I have also been able to clarify the limits imposed on their size. The primary limit appears to be a maximum total size of around 32,650 bytes (but slightly less than 32,767) available for the storage of xattrs for each file in iCloud Drive, rather than any lower limit on the size of individual xattrs. If the total size of xattrs for any given file exceeds that, then individual xattrs are removed, starting with the largest, until the total falls below that maximum.


Update (2023-08-04): Craig Grannell:

My iCloud Drive has suddenly started restoring months-old folders but without whatever was in them, which at first freaked me out because I thought I’d lost a load of files. Why is it doing this? No idea. On the plus side, no catastrophic data loss this time.

Exploring Unicode in macOS With Clui

Joel Bruner:

My new tool clui, pronounced “clue-ee” offers Command Line Unicode Info with the ability to export to a variety of formats like CSV, JSON, YAML, RTF and more. While I’ve written a few macOS command line tools geared to the Mac Admin like jpt the JSON power tool, ljt the little JSON tool, shui for easily adding AppleScript dialogs to your shell script, and most recently shef a Unicode text encoder and formatter for shell scripters. This is one is almost “just for fun” although you might find some practical uses for it. Writing shef opened my eyes to the stunning amount of detail and craftsmanship in macOS’ Unicode-aware fonts, which comprise not just the alphabets of the world but signs, symbols, and even Egyptian hieroglyphics! While macOS’s built-in Character Viewer does a pretty good job to group and display these characters it’s a painstakingly manual process if you want to get info on a range of characters. I hope clui makes it fun and easy to poke around the vast Unicode neighborhood.