Monday, January 24, 2022

Increased iCloud Errors

GoodNotes:

HTTP 503 is a temporary error code (“Service unavailable”) indicating iCloud servers aren’t responding correctly to requests from your devices. The error typically gets resolved as GoodNotes automatically retries, but we’re getting many reports of the error lingering on, causing sync failures.

This issue is not apparent to us and we’ve escalated the case to Apple Technical Support team for investigation. It seems it’s happening to other apps as well.

My devices haven’t reported any error codes, but iCloud Drive and Photos have been particularly unreliable and slow for the last week or so. As far as I can tell, eventually all my data did sync. Others, though, have reported losing records and files.

Chad Etzel:

well, add me to the pile of ppl making an iCloud status dashboard for my app

Paul Haddad:

iCloud errors seem to have really increased over the last couple of days. Wonder how many devs have to add these for Apple to I don’t know fix the problems or at the very least publicly acknowledge them.

Craig Grannell:

So pissed off with Apple’s iCloud today. Wasted almost an entire day to it throwing up 503 errors. My backup Apple ID is fine, so it seems like my main ID has been rate-limited for some reason. Bloody infuriating. So much for “it just works”.

RawMean:

iCloud Drive returns a 503 error (service unavailable) since Nov. and it’s affecting all apps that use iCloud Drive. @AppleSupport has not done anything about this and more AppleIDs are being affect.

Adam Overholtzer:

Supposedly they are aware of it, but it’s been going on for weeks — there are forum discussions going back to early December if not earlier.

MeanRaw:

DTS did confirm that the problem is on the server side. But they said it’s outside their jurisdiction and provided no ETA. This started in November

Craig Grannell (tweet):

Then Soulver failed — suddenly and very badly. I needed to restart my iMac so was shutting down all my apps. Soulver threw up a permissions error. A week of input was wiped out in an instant. This was a shock on multiple fronts: in part because of the data loss, but also because Soulver is one of the most robust apps I use. It had never failed before. […] The culprit was clearly iCloud.

[…]

A quick look on Reddit had already suggested to me that the problem had in fact been around for months, rather than days; terrifyingly, Zervaas said he’d seen this issue “on and off since May 2021”, and his own app only currently fully works on some of his Apple IDs.

Becky Hansmeyer:

Ok, there’s clearly an issue going on with iCloud sync right now affecting a lot of users across many different apps. What’s the best way to report this? Radar? I haven’t made any changes to my app since October and users have just recently started reporting sync failures.

Juli Clover:

iCloud failures are a major problem for app developers because the end user of the app doesn’t know that it’s an Apple issue, so customers blame the app developers for a non-working sync feature that they have no control over.

Malcolm Owen:

Outside of Apple Technical Support queries, there has yet to be any official confirmation of an issue from Apple, that it exists or is being actively investigated.

The System Status page is useless, as always.

Becky Hansmeyer:

This is an Apple screw-up of discoveryd proportions.

By the way, I wonder what’s going on with discoveryd these days. Presumably, mDNSResponder was being rewritten for good reasons. Is it slowly being refactored to address the original motiviations for discoveryd? Is a new discoveryd-style rewrite in progress? Did they just decide to leave this code untouched?

Previously:

Update (2022-01-24): Steve Troughton-Smith:

Developer sound off: if you’ve recently been struggling with iCloud sync issues in your apps that sound like they may be related to today’s news, can you reply with whether you’re seeing this with 1) iCloud Drive, 2) CloudKit + your own data model, or 3) CloudKit+Core Data

Craig Hockenberry:

I hate to say this, but iCloud can’t be trusted. It’s disheartening because there has been improvements in the past couple of years.

[…]

The most infuriating part about this is Apple’s complete silence in public.

Privately, they are telling developers there are issues, which is a step in the right direction.

But it’s our customers who are losing the fucking data.

Update (2022-01-25): Rod Christiansen:

I had an iA Writer document literally revert back in front of my eyes to a version from 5 minutes ago!

Happened twice today.

MacJournals:

Analysts could do worse than asking about things like this during Thursday’s conference call.

If services are Apple’s future, how can these problems persist for months? Why is iCloud designed so neither devs nor users can get answers or escalate?

Chris Vasselli:

I believe the key is handling rate limiting properly, which CloudKit (erroneously) returns as 503 errors. But there are some gotchas to watch out for:

[…]

The suggested retry period is too short, and repeating your request after that period will trigger rate limiting again (seriously).

[…]

If there were multiple errors, they will be returned as a single error with a partialErrorsByItemID value containing the underlying errors. The parent error will not contain the retry information, you need to check the underlying errors.

I have seen certain errors (specifically zoneBusy and requestRateLimited) actually omit the CKErrorRetryAfterKey value, and so I have a default value of 30 seconds in case it is missing.

Quota exceeded errors (meaning the user is out of iCloud space) seem to erroneously include CKErrorRetryAfterKey values, even though retrying them is not actually going to help.

[…]

After reading more, it looks like there is more going on here than just properly handling the retry timeout. Some accounts are getting 503’s on all requests, no matter what.

Update (2022-01-26): Becky Hansmeyer:

Regarding the recent iCloud sync issue, I received a very kind and detailed response from Apple indicating that the underlying issue causing the 503 errors/sync failures has been fixed!

Adam Overholtzer:

Will I get an email? What about an apology? What about a removal of all the 1-star reviews this nonsense has earned me the past few weeks?

Seems like this issue, affecting essentially the entire platform, unfolded and was eventually resolved with no official communications from Apple acknowledging the problem or fix.

Paul Haddad:

The number of iCloud Sync is broken reports we’ve gotten over the last 2 days has gone down to basically 0.

See also: MacRumors.

Update (2022-01-31): Jesse Squires:

But as a developer, the iCloud issues are even more frustrating. Sure, we all know software is hard. Minor, intermittent issues are to be expected. But two things are unacceptable — data loss and the (still!) terrible communication from Apple.

If Apple fails to maintain quality software and prevent data loss — that affects our users and there isn’t much we can do about it. Apple strongly promotes these services. Apple really wants third-party devs to integrate iCloud support into their apps. And yet, they seemingly can’t be bothered to make it reliable. Among the many other issues that developers have with the App Store, this ranks at the top. Third-party developers must pay 15-30 percent of their wages to Apple. And for what? What are we paying for? A totally broken iCloud? Developers that depend on iCloud get doubly screwed.

And then, to make matters worse, developers received zero communication from Apple during this outage — not even an acknowledgement that an issue existed.

A few days after the supposed fix, my iCloud Drive uploads broke again.

Craig Grannell:

Same here. iCloud sync for apps is now OK here, but iCloud Drive is totally dead. Nothing is uploading. FFS. This is ridiculous.

Previously:

Update (2022-02-04): Howard Oakley:

iCloud provides users with one real control: the on-off switch, which isn’t in the least helpful when the problem is failure to sync. My free utility Cirrus can perform a basic test of the first of the three services, and gives access to iCloud entries in your Mac’s log. But those only help in diagnosis, not in treatment.

Cirrus does enable you to download files in the first service category (and you could also use this with shared files used by third-party apps). That enables you to ensure that you keep a local copy, which will also be backed up by Time Machine and other means of local backup. But it does nothing to help you sync those documents with or through iCloud when the service is faulty.

Update (2022-03-16): Adam Overholtzer:

Just say no to CloudKit! My app has 1000s of 5-Star ratings but the stupid Error 500 thing got me enough bad reviews that I’ve dipped below a 4.7 average for the first time. I also never got a word about it from Apple, despite a DTS report and being quoted in a couple articles.

Update (2023-12-19): Adam Overholtzer:

I’m getting another surge of email about iCloud/CloudKit sync errors, specifically Error 500, unspecified internal server error.

This seems to happen every year around this time, except this year it already happened back in August.

Lots of developers are reporting this lately.

5 Comments RSS · Twitter

Happening since December is not November... But yes, again it needs some bad PR before Apple do anything.

Not clear to me why discoveryd was even necessary; best I can tell it had a few features for introspection of state, but that's all. mDNSResponder works, is fully functional, is well-tested and cross-platform, and its replacement would probably be another massive turd. I hope they leave it well alone!

I completely removed iCloud sync integration from my apps because so many users had problems.

Maybe now Apple can work on figuring out how to sync Safari iCloud tabs reliably.

Somebody with macOS 12 on M1 can maybe confirm reference to discoveryd found here: https://github.com/canonical/multipass/issues/2265

Leave a Comment