Kindle Annotations Lost in Book Update

Its quite impressive that this is the first time you've suffered at the hands of amazon. The cloud seems to be a learning experience for many a company, bringing unexpected black eyes with it. May you go another 15 years before the next one. May we all!

I don't think Amazon failed you, I think your backup system failed you.

I lost a lot of notes reading Kindle books on the iOS app, even without any updates. Not a huge deal, I guess, but if I'd been a student it would have been devestating. I don't think they take that feature very seriously, and it makes the Kindle not ready for the education market.

@ryan

Michael may have (or maybe should have) had a backup of his annotations. But even if that were the case, such a backup would exist - as all exported Kindle notes do - as a plain old text file. The actual highlights, bookmarks, and notes as placed on the book's electronic pages can't be backed up - or restored - as far as I know.

@ryan

Both failed. While it is complete common sense that "I should have a local copy of anything important to me". I agree 100% with this sentiment.

However - when the cloud loses my data, the cloud failed. The purpose of the cloud is to not lose my data. I *should not* have to keep a backup of everything I store with a cloud service. The commonly accepted fact that one should do this in self-interest is a wise one, yet it is not an excuse for a cloud service failing.

If you're using a Mac, I concocted a little Applescript that will allow you to export your notes and highlights to OmniOutliner, OPML, or Evernote. I haven't updated it yet for Lion (though it might still work), and it runs really slowly when you have a lot of notes. Nevertheless, you may find it useful to making a long-term archive of your annotations. You can read more and download it from my blog The Savvy Technologist (http://technosavvy.org/).

I've got some plans to make sure it works on Lion (again, it might now) and runs a lot faster. I'll post info on that update when it's available on the blog too.

@Tom My experience with Amazon has been better than with probably any other major corporation. When they screwed up—e.g. sent incorrect or damaged items—they always fixed it promptly. A few times I screwed up—e.g. double-purchased a video or bought the wrong item by accident—and they handled that gracefully, too.

@Ryan True enough. I regularly back up my Kindle annotations from the Web site. I should probably do so more often. It’s great to be able to copy and paste the text and click the links to the context of the notes. Overall, I see this as a plus for the Kindle platform. With iBooks, all you can do is take screenshots of your annotations.

I thought maybe the annotations would be in my Safari Web history, but it looks like it doesn’t save the full page contents for https pages.

@Dan Indeed, I just connected my Kindle to the Mac and found that the text file at:

/Volumes/Kindle/documents/My Clippings.txt

still contains my highlights from the book! I had assumed that this file was updated with the sync, but I guess it’s too low-tech for that. So there’s a loss of fidelity, but I got my notes back. (The Kindle support folks should have pointed out this possibility, I think.)

@Tim Thanks for that! I'd love an equivalent for iBooks & .epub based annotations. Know of any out there?

I think this has to do with how indexing for positioning data (bookmarks, annotations, etc) takes place.

If you were to annotate a section that were removed in a content update/correction, you wouldn't be surprised if that annotation went away or got orphaned.

If annotations are saved based on relative position (3,512th word in the book, Chapter 2 - Paragraph 4 - Word 1, etc) then if a single word/element is different or in a different order between versions of an eBook, someone (the eBook publisher, Amazon) would have to create a lookup table to map old position data to new position data.

Hopefully that process will be automated one day, but I believe that's your culprit. It's a lot easier to just throw that data away when a correction needs to be pushed out rather than map all the old annotation data to the new.

I agree that this sucks, by the way :)

I purchased several computer programming reference books in ebook versions 5 years ago from Amazon. Turns out that they were only viewable on the pc at my office, and not transferrable anywhere else on my next office pc or home pc. I had emails regarding the purchases and they knew I purchased them but did nothing to make my books readable again. I've never bought any more ebooks from them since. I have their Kindle Reader on my phone, but I just don't trust them to look out for the interests of the customers. I haven't bought any other media from them in the past two years either, and I am a voracious reader of 200+ books a year. I am using the library more and reading my backlog of printed book purchases/gifts. Hope to convert all my keeper printed books to digital in the next 5 years or so, when the ecosystem is more stable and backlist items are available. I want something like iTunes match for my existing print library. Amazon is just out for a buck, and in a low margin bookselling business, I don't trust that they will be around for decades to manage my book collection when their proprietary Kindle software needs to be revised/rewritten for the next big thing. I've bought numerous books and cds from Amazon, but have they offered me digital versions of those purchases? They just want to sell you stuff over and over again.

@Art Certainly, this is a technical challenge, though it should be solvable given that only a tiny fraction of the book text changed. The core problem is that Amazon sent out an e-mail saying that it would work. Not only did it not work, but apparently the rest of the company knew that it wouldn’t work. Had I known up-front that they hadn’t solved the updating problem, I would have been happy to stay with the old version of the book.

Kindle has a very simple, almost stupid system for storing notes, underlining, and even keeping track of where you are in a book. I discovered that Whispersync only keeps the furthest location in the book you've read, as opposed to actually tracking the page you were reading at the last time the book was opened. It's fine if you read a book from beginning to end, but if you have a book with endnotes and go to one of those endnotes, Whispersync will now try to sync all of your Kindles to the endnote and not to the last place you were actually reading. So you basically have to remember to bookmark a book with endnotes when you finish reading to transfer your place.

This isn't really a cloud problem, it's a bad email problem.

When Amazon has offered updates of books I've bought on the Kindle, their email has specifically pointed out that notes etc. would *not* be preserved, e.g.:

An updated version of Reamde: A Novel (ASIN:B004XVN0WW) is now available. It's important to note that when we send you the updated version, you will no longer be able to view any highlights, bookmarks, and notes made in your current version and your furthest reading location will be lost.

The truly odd thing is that they apparently now have someone sending out those messages who doesn't understand how this works.

Highlighting some interesting passages? I had over 350 highlights and notes. You can bet I was freaked.
Then I opened my Amazon Kindle app for Mac (which recently got a beautiful new update, btw), and saw them there... waiting for me.

Well done, AMZN.

Kindle highlighting is also a resource hog. I tried it on a very large book, using my normal highlighting/underlining methods, and found that after a few hundred pages, response and refresh times lagged - it was actually slowing down my ability to read. I had to go back and delete all the highlights to get acceptable performance.

If I need to highlight or annotate, I now use a PDF - or paper. Codex technology prevailed for centuries for good reasons!

I've thought I lost notes before, but I've always got them back, one way or another. One thing to remember is that customer support people at Amazon (and elsewhere) don't always have the final answer. These guys can't keep up with everything. Sometimes it helps to call again or ask for a supervisor.

Anyway, the last time I lost notes (with the Steve Jobs update), I called customer support and wasn't able to resolve the problem, but something the guys said made me later delete the book and then pull it down again from the Archives folder. The notes reappeared.

You should also be keeping your Wifi on enough for the backups to happen, and enable those in the preferences. And at Kindle.amazon.com you can find your most recent notes for the book online.

[...] As a contrast, the Kindle Fire didn’t even come with a connection cable (just a charger). I guess Amazon is backing up/hosting media in the cloud for me, no idea about backups of apps/settings. If I had more data on it I cared about, I’d be worried. [...]

@Art - I don't doubt there are significant technical hurdles for Amazon, but surely no matter how difficult the problem this is one they have to solve. What if video games deleted your save data every time you updated your game? What of I lost my contacts every time I updated my phone?

The complexities of ebook data pale in comparison, and Amazon looks downright silly here.

@Dan - it may not technically be a cloud problem, but if users start associating data loss with cloud storage...

Last night, I went through the “My Clippings.txt” file with BBEdit and recreated my annotations via the Kindle Mac app.

This afternoon, I received an unsolicited call from Amazon’s Seattle office. The support person appologized profusely, and she said in effect that everything I had been told before was wrong. First, the e-mail I received was correct: Amazon does claim to automatically transfer annotations to the updated version of the book. Second, in the event of a problem, it is possible for them to recover annotations from the backups. All you have to do is ask. She said that Amazon had gone ahead and done this for me.

I checked my book and found that the manual restoration that I’d done had been overwritten by the auto-updated annotations from Amazon. This was a problem because the transferred annotations were inaccurate. Many of the highlights started or ended several words too early or too late. Some highlights had been split in two. Others were blank or contained only one word. I was told that it’s expected that the annotations will be slightly shifted or off after a book has been updated. It seems that the locations are stored in a very rudimentary way. Based on this experience, I would advise not updating any books that you’ve significantly annotated.

I asked that Amazon restore the backup from when I had manually added the annotations to the updated book. I was told that this wouldn’t take long, and that they’d follow up with me, definitely today, and probably within an hour. It is now 6.5 hours later, the backup has not been restored, I’ve not heard anything, and there’s no answer at the phone number.

This morning, I found that a good portion of the incorrect annotations had been fixed. Soon after, I received an e-mail from Amazon saying that they’d restored the backup as requested. I'm not sure why the restored annotations were still slightly off—several blank highlights and extra words added at the ends of some of the highlights—but it’s certainly a big improvement. I corrected the highlights in the Kindle Mac app and am considering the problem solved.

Interesting saga in the last two comments.

Amazon is a rarity in that they're essentially the only big tech company where I actually enjoy being a customer these days.

You can't totally trust their cloud services because you can't totally trust any cloud services. But at least I feel like they're focused on their customers' satisfaction in a way that is a rarity these days among big companies.

"Based on this experience, I would advise not updating any books that you’ve significantly annotated."

Even though things seemed to get mostly worked out in the end here, I'd say that's good advice in general. No matter how good their procedures get, there are just too many places where things could go wrong in such a scenario.

"and there’s no answer at the phone number."

That's no surprise, no? Amazon doesn't do phone.

@Chucky Agreed. Apple is hit or miss. Sometimes they’re very helpful. But one time they damaged my notebook, blamed it on me, and wanted to charge me $1,000+ to fix it before they would give it back. (They eventually relented, but didn’t fix it.) Another time, multiple support people told my dad what a moron he was that the iPhone 4 antenna wasn’t working for him (even in a bumper case) and made him drive hundreds of miles to return it and reactivate his original iPhone.

Amazon does have an option to get support via phone. You can also choose e-mail or chat.

[...] And then there are the problems with updating books. [...]