Lost Calendar Events, Change Notifications, and iCloud
Google Calendar initially did not seem to be fully reliable with iOS, so I’ve been using iCloud’s calendar pretty much from the beginning. It worked great until about a month ago. Since then, I’ve lost at least four calendar events, in a strange way. I would edit an event’s title or note and see this reflected on the calendar. Then, within a day or so, the event would disappear from all my devices. The only reason I noticed this is that, because of the recent edit, it was fresh in my mind that I had an upcoming appointment.
My first thought was that maybe I had deleted the events by mistake. The events were on a shared calendar, and my wife had subscribed to receive e-mail notifications of any changes that I made. She had an e-mail for each edit, but there were no records of the deletions. So it doesn’t seem like I deleted them. She wouldn’t intentionally delete my events, and if she had done so by accident I should have received notification e-mails, which I didn’t.
Could it be due to a client bug or error? I don’t think so, based on my rough understanding of how CalDAV works. Plus, an accidental delete command should have generated a notification e-mail from iCloud, anyway. I use Fantastical on all my devices, so I contacted Flexibits Support. I got an immediate reply, but they had not heard of problems like this and agreed that it didn’t seem like Fantastical was responsible. Calendar clients don’t directly add and delete shared files; they send streams of commands to the server. So even if the wrong command were sent, the server should still have sent a notification.
Step two was that I wanted to make sure I wasn’t missing any other calendar events, ones that were not fresh in my mind. I archive my old calendars to PDF, so I know I have a safe copy of everything from prior years. But I certainly don’t want to miss something that’s coming up.
Apple’s tools don’t seem to provide a good way to answer this question. Calendar does not support browsing using Time Machine. The Web version of iCloud did recently add a way to restore previous calendar versions, but this is not a good fit for several reasons. First, there is no way to download or browse your calendar backups, much less compare them to find the differences. You have to overwrite your current calendar—actually all of your current calendars, and reminders. Second, restoring a backup is problematic. It warns you that “all sharing information will be removed” and that “scheduled events will be cancelled and recreated, and invitations reissued.” Plus, it wasn’t clear to me what to do. How far back should I restore? How exactly would I find deleted events? Reapplying all the changes from the backup to now would be practically impossible, so I would actually need to restore again, from the most recent backup, and then try to recreate just the deleted events. If there were a problem with the backup or restore I could end up in a much worse situation. I didn’t like this approach.
A better idea seemed to be to dig into the files myself. I had backups from my Mac and could compare the files if needed. But, even better, I have been making regular Git archives of my local Calendar store, which is essentially a folder of .ics files, one per event. I was able to look through the Git logs and see which files had been deleted. They were all ones that I had edited recently. I recreated the events manually in Fantastical.
Step three was to try to set up better logging for the future. If I could easily see a log of the changes that I had made, it would help me to notice anything missing, and I would also have a backup, of sorts, that could be used for easier manual reconstruction. iCloud’s e-mail change notifications would make a pretty good log, but they only notify you of changes that other people make. To get a complete log of changes, I needed to create a separate Apple ID that would only be used to receive notifications. It turns out that doing this from the iCloud Web site doesn’t work because you won’t have access to Calendar, only to the iWork apps. I had to sign into the iCloud account from a Mac OS X installation in VMware in order to activate the Calendar features. Then I could access Calendar from the Web. (The e-mail notification option is only in the Web version, anyway.)
At this point, I thought I was done except that, a day or so later, the notification e-mails stopped. I had not touched that iCloud account since enabling the notifications, so I know that I didn’t change anything. However, when I logged in again I found that “Email me shared calendar updates” was mysteriously unchecked. I checked it again and since then the notifications have worked as desired. Fingers crossed that this continues.
In conclusion, iCloud Calendar is not as reliable as I thought it was. This is probably not the type of reliability problem that would show up in Apple’s statistics. Indeed, I did not report it to Apple because I did not know how to do so in a useful way. Contacting Apple’s support didn’t seem like a productive avenue, and since my Mac is more than three years old it’s no longer covered by Apple Care, anyway. So I’m hoping this was just a fluke that won’t happen again. More generally, Apple seems to be trying to get rid of the notion of files, but the tools for dealing with non-file-based data are few and crude. And this all becomes more complicated when the data is shared among multiple users.
The details have shifted over the years, but the basic form remains the same: [Notes] Changes made on one device sometimes don’t show up on others. The situation improved somewhat since Apple introduced CloudKit and started migrating apps to it, but problems still occur. It happened to me just recently: I added a note to the Apple Notes app on my iPhone, then switched to my Mac, where Notes was already running. No note.
[…]
I get where Apple’s coming from: They want sync to feel seamless and effortless, something that happens without the user having to think about it. But the fact of the matter is, it doesn’t work that way. Sync rarely does, because it’s a very hard problem to solve. Apple isn’t alone in having trouble with it by any measure. But by not giving users a way out when problems do happen, they increase the perception that Apple services don’t work well.
iCloud backups from iOS are broken. Again.
If you think your phone is backing up to iCloud, you should check. When it fails, it often fails silently, and for weeks at a time.
Don’t trust iCloud backups on your iOS devices.
See also: Siracusa’s Pages lament.
Cloud services, meanwhile, are still less reliable than Apple’s competition, and the integration — Apple’s supposed strength! — with Apple’s software is at best a source of irritation and at worst very worrisome from a security perspective: little things like constantly being prompted to enter one’s password are not only annoying but also corrosive when it comes to what should be a healthy skepticism about sharing the keys to your life.
The problem in all these cases is that Apple simply isn’t set up organizationally to excel in these areas[…] The root problem in all these cases is the lack of accountability: as long as the iPhone keeps the money flowing and the captive customers coming, it doesn’t really matter if Apple’s services are as good as they could be. People will still use the App Store, Apple Music, and iCloud, simply because the iPhone is so good.
Update (2016-05-10): Russell Ivanovic:
Tried to sync a desktop background over iCloud Drive. On the other side: “Can’t open Document (null)”. Can’t copy the file either.
Deleted the PNG, added it again, same thing. Gave up and put it in DropBox, works fine.
Update (2016-05-11): John Chaffee informed me that BusyCal’s log of changes from the server could help spot deletions. BusyCal also has an automatic backup feature that can restore a single calendar and does not have to overwrite existing calendars.
Update (2016-05-12): Hwee-Boon Yar:
I have had events that were around for a while mysteriously duplicated. When I delete one of them, both disappeared.
Update (2018-05-01): I’m continuing to have problems with iCloud Calendar not consistently sending e-mail change notifications.
7 Comments RSS · Twitter
"In conclusion, iCloud Calendar is not as reliable as I thought it was."
Michael...
My sense, perhaps wrong, is that Apple is making changes in their backends in preparation for what gets released at WWDC. While I've not noticed any calendar issues I have noticed issues in other services including Siri. The latter in particular seems less accurate than it was a month or two ago in terms of basic speech recognition.
I should add that while I have some problems with Apple services, I have problems with all services. The infamous bug in Dropbox where some file keeps syncing non-stop popped up again for me last week. Google clients have the past year or so been getting worse after a long period of improvement. I finally uninstalled Google Drive because of stability issues.
That's not an excuse mind you. All these companies should work hard - especially regarding data loss. Just that I think people judge Apple more harshly than the competition because they are Apple. We expect more from Apple. Add in that Apple's backend services have sucked so bad so often (Mobile Me, the infamous first year of Siri or Maps, etc.) that we see it through a somewhat myopic eye.
My guess, again perhaps wrong, is that the biggest problem with Apple services is applying the same mindset they use for OSX and iOS to services. Services should be constantly improving and not be so tied to the yearly WWDC release schedule. What helps marketing for the fall OSX and iOS releases really hurts Services I think.
"I should add that while I have some problems with Apple services, I have problems with all services."
Sure. But IMHO, this a narrower issue than "services". The issue is "sync". Apple is notoriously incompetent at sync. Have they ever produced a sync service that wasn't a disaster? To the best of knowledge, no. And I know sync is hard™, and that all service providers will have occasional glitches. But Apple really does seem to be in a class of its own in their complete inability to do sync without mangling user data.
I was gently poking fun at Michael upthread for entrusting his personal data to an Apple sync service, since that seems to violate Best Practices. I know devs, and folks who wrrite professionally about Apple, must make themselves familiar with these things. But I've always respected Kirk McElhearn's approach. He's gotta write about iTunes, so he's gotta enable iTunes Match and Apple Radio. But understanding that Apple Can't Do Sync, he has a dual setup, with his personal music data in a setup with iTunes Match and Apple Radio disabled, and thus safely protected from Apple's inability to do sync...
@Chucky I seems like I should know better, but in this case I had a previous bad experience with Google Calendar, and Apple was actually one of the inventors of CalDAV so it seemed like they would know what they were doing. Are people having better luck with FastMail or ownCloud these days?
I haven't tried FastMail calendar syncing yet because iCloud calendars have been rock solid for me for years. That and contacts have always been two bright exceptions to the unreliable services Apple otherwise offers.
Maybe I should start testing calendars with my FastMail account. These types of issues are tough though, because I can't realistically port all my calendar data over and then spend the next year wondering if I happen to notice the _absence_ of events I'll likely have forgotten I added to the calendar.
iCloud disaster!!!!!
Two weeks ago I decided that sense I had so much extra space for backups in my iCloud account that I would go ahead and turn on the "Calendars" feature on my iphone 6s. Well I did indeed and Poof all of my calendar events dating back 8 years vanished and on ALL devices. So I called Apple support and the tech said no worries you have a backup on iTunes, but you should do another backup before you restore. I did as he suggested and Poof the backup from the previous day that I could have restored vanished!
That was 11 days ago. Apple "Engineers" have supposedly been working on finding my lost calendar events for 11 DAYS!!!! Right??? The .pst files I have from my backups on my laptop and carbonite have only two of the calendars saved and they are not the ones I ever use. I've looked for the .ics files, etc. etc. Apple is a disaster and I can't say much for Microsoft either.
Apple support is a total joke. Every time they make a change we as customers suffer. Where is a company who can actually get the job done and be reliable? Any tips out there?????
Thanks
[…] Calendar: This works better (with iOS) than Google Calendar. I had some data loss last year, but otherwise it’s been […]
