Apple Music Deletes Personal Music Files
James Pinkstone (via Hacker News):
What Amber explained was exactly what I’d feared: through the Apple Music subscription, which I had, Apple now deletes files from its users’ computers. When I signed up for Apple Music, iTunes evaluated my massive collection of Mp3s and WAV files, scanned Apple’s database for what it considered matches, then removed the original files from my internal hard drive. REMOVED them. Deleted. If Apple Music saw a file it didn’t recognize—which came up often, since I’m a freelance composer and have many music files that I created myself—it would then download it to Apple’s database, delete it from my hard drive, and serve it back to me when I wanted to listen, just like it would with my other music files it had deleted.
[…]
What Apple considers a “match” often isn’t. That rare, early version of Fountains of Wayne’s “I’ll Do The Driving,” labeled as such? Still had its same label, but was instead replaced by the later-released, more widely available version of the song.
[…]
Should I choose to reclaim my songs via download, the files I would get back would not necessarily be the same as my original files. As a freelance composer, I save WAV files of my own compositions rather than Mp3s. WAV files have about ten times the number of samples, so they just sound better.
[…]
Amber relayed to me that she’s had to suffer through many calls from people who cancelled their Apple Music subscription after the free, three-month trial, only to discover that all of their own music files had been deleted and there was no way to get them back.
So my files were temporarily restored; but the only way to prevent this from happening over and over, according to Amber, was to cancel my subscription to Apple Music (which she herself doesn’t use due to the above-listed reasons) and to make sure my iCloud settings did not include storing any music backups.
Update (2016-05-05): Warren L. Habib:
All true. Happened to me and same reason I don’t use Apple Music tools at all.
Rosyna Keller says that iTunes moves the files to the trash but doesn’t itself delete them and notes Apple’s documentation on the subject:
When Apple Music adds these matched songs to your iCloud Music Library, Apple Music doesn’t change or alter your original music files that reside in iTunes for Mac or PC or on your iOS devices from which they were added.
[…]
Songs that we find that are available in our catalog are instantly accessible on all of your other devices, and you can save them for offline listening on your devices indefinitely. When iTunes Match adds these songs to your iCloud Music Library, iTunes Match doesn’t change or alter your original music files that reside on iTunes for Mac or PC or on your iOS devices from which they were added.
This explanation of how it’s intended to work seems to contradict both Pinkstone’s report and the statements of Apple support person Amber. It may be that Apple’s documentation is wrong, that Pinkstone is mistaken about what happened and what he was told, or that the software is not working as documented.
Songs that can’t be matched are uploaded from iTunes to your iCloud Music Library and stored in iCloud in their uploaded form. You can also download them in that same uploaded form when you save them offline on your other devices.
It does not say what form it uploads them in, though the iTunes Match FAQ says that it supports WAV.
Rosyna Keller notes Apple’s other documentation:
Songs encoded in ALAC, WAV, or AIFF formats will be transcoded to a separate temporary AAC 256 Kbps file locally, before they’re uploaded to iCloud.
Previously: Apple Music: Problems Adding Albums, Songs Deleted.
I’ve got a 103GB music library, including that original Fountains of Wayne song, in iTunes library, attached to Apple Music. All fine.
What that piece is right about is Apple Music’s terrible matching and inability for the user to override/replace bad matches. It’s painful.
My theory is that this dialog box is causing a fundamental misunderstanding of what happens next.
iCloud Music Library is confusing as all hell, and it’s been so since the first days of iTunes Match. That’s absolutely Apple’s fault.
[…]
So no, Apple Music did not mysteriously delete Pinkstone’s library, and it won’t delete yours. That’s simply not how the service works on the Mac. But if you’re not aware of how iCloud Music Library stores copies of tracks, you may delete your local copies to save space, thinking you can get them back — and get screwed as a result.
It’s possible there is also a bug in iTunes that is deleting files. It may be just confusion, or there may also be another vector?
My takeaway from all the people commenting is that no one seems to fully understand how it’s intended to work, or whether it works as intended, so it’s best to stay far away.
Update (2016-05-06): The comments on Caldwell’s piece show the same divide that Pinkstone’s post noted:
I’d already visited the online forum, I said, and they were no help. Although several people had described problems similar to mine, they were all dismissed by condescending “gurus” who simply said that we had mislocated our files (I had the free drive space to prove that wasn’t the case) or that we must have accidentally deleted the files ourselves (we hadn’t).
As Caldwell explains, Apple surely didn’t intend to “steal” anyone’s music, which is why I didn’t use the original headline. But I would not make the leap from defense of Apple’s intentions to assuming that the users who say their files were deleted are at fault or mistaken.
It sounds like either a severe user error or a severe bug, and there’s no way to know which it is, but regardless, this aside is good advice[…]
Update (2016-05-07): Ben Lovejoy:
I was extremely unamused when, less than a month later, I boarded a 10-hour flight, got out my iPhone and prepared to listen to some music only to discover that hardly any of it was there. Of all my favorite albums, lovingly selected as the ones I wanted to have available at all times, just a handful were still present. Others claimed to be present but in fact had just one or two tracks. Everything else had the cloud download icon next to it – which did me no good at all at 37,000 feet.
The iOS Apple Music app had, entirely arbitrarily and without consulting me, dumped most of it.
The poll in the comments says that 72% of readers have had local music disappear.
Update (2016-05-09): Juli Clover:
Apple Music users with personal music collections should create a backup on an external hard drive, which will ensure no music ever goes missing through user error or an Apple Music bug.
Amber is wrong. Neither Apple Music nor iCloud Music Library deletes music files. This simply doesn’t happen.
[…]
I don’t know exactly what happened to this user. I contacted him by email trying to get more information, and he told me that he no longer uses Apple Music, so he really can’t help elucidate the issue. There are a few hypotheses circulating about what may have happened, and none of them make total sense. Something deleted his music files—including music he composed—and it’s hard to figure out what was responsible.
People normally sold on more or less everything Apple – people, indeed, through which Apple leak their event preannouncements – have tried and simply given up. The thing eats people’s files. I keep backups and I avoid turning on iCloud Music Library, so I thought I was safe, but in the year I’ve been using it, it’s been pulling down duplicates of things I bought on iTunes ages ago and inserting them as new entries at the top of my library (sorted descended by Date Added). On iOS, it has “very helpfully” restored some ugly-as-fuck album art that I “Got Info” on ages ago and deleted the album art from. And just a few days ago, it seems to have re-checked a bunch of albums and songs that I for various reasons have unchecked through the ages. (Every song in iTunes has a checkbox, and if you uncheck it it won’t be played unless you manually start playing it. If iTunes is to Winamp/VLC what Excel is to TextEdit/Notepad, imagine silent data loss in the form of formatting, value or formula changes in spreadsheets. Heads would roll.)
Update (2016-05-10): Ruffin Bailey:
iTunes Match should’ve let Apple know that they weren’t doing a great job of providing its eponymous function -- matching -- and they really needed to make sure they could get 1.), 2.), & 3.) right before going whole hog into a unified Apple Music. But it’s still just a database management problem. They “simply” should have been much more defensive with Apple Music for when matching failed. If your user doesn’t have a file three places, don’t delete it.
But as long as you have a space for a flag on each file to say which it is -- an unDRM’d original*, an unDRM’d “likely match”, or a DRM’d file that’s never been matched -- you don’t have this trouble.
Update (2016-05-13): Serenity Caldwell:
Based on several Apple Support threads, it appears that the most recent version of iTunes 12.3.3 contains a database error that affects a small number of users, and can potentially wipe out their music collection after the update. The error has been mentioned a few times, primarily on the Windows side, in the weeks since the 12.3.3 update, but appears to be rare enough that it hasn’t previously received major press. Apple did put out a support document shortly after the 12.3.3 update that walks you through some fixes if you find that your local copies of music are missing.
I can’t state for certain that Etropolsky and Pinkstone fell victim to this bug, but based on their descriptions and screenshots, it seems likely that the following happened[…]
Update (2016-05-14): Serenity Caldwell:
Apple on Friday confirmed to iMore that a very small number of users have reported seeing issues with iTunes removing their locally-stored music library. The company has yet to be able to reproduce the error, but plans to release an update to iTunes next week in hopes of fixing this issue and reducing customer confusion.
Serenity Caldwell (via John Gruber):
I read it as “we’re still not convinced it isn’t user error, but we’ll make the dialog boxes less terrible.”
There’s no log in iTunes recording what you add and what you delete, no way to look back at previous versions of your library to compare it with the present. However, if you use Time Machine to back up your media, there is a way to see what has been deleted during each backup.
[…]
When you run Backup Loupe, and scan a backup, it takes a while to compare all the files to the previous backup, and you can then examine the differences. Here’s an example. Backup Loupe indicates deleted files by a red number in parentheses. You can see that on my Music disk, 35 files were removed; I drill down in the folder structure to see which files those were.
Update (2016-05-17): Kirk McElhearn:
Plenty of people delete individual files, and if there’s a bug in iTunes that causes the app to trash all your music when you delete one file, that rules out some very unlikely behavior by users. I have no idea if this is what’s going on with deletegate, but it’s certainly worth looking into.
Update (2016-05-18): James Pinkstone (Hacker News):
Tom, along with his boss Ezra, had just spent most of Saturday at my dining room table with me, trying to recreate a disaster like we were Netflix green-lighting Fuller House. So far, no luck.
In the days leading up to our face-to-face encounter, they’d earned more of my trust when they acknowledged that A), they’d read the phone transcripts, and although they maintained that she was mistaken, they did not dispute my account of what Amber had told me, and B), they, too, were convinced this was not user error.
Via Chance Miller:
The Apple engineers hooked up an external hard drive to Pinkstone’s laptop and ran a specialized version of iTunes trying to recreate the mass deletion of music. They spent most of Saturday running tests and communicating heavily with Cupertino throughout the process. On Sunday, Tom returned to Pinkstone’s house to collect data of his iTunes and Apple Music usage on Saturday night, hoping to find some correlation between his habits and the deletion of music. But ultimately there was nothing.
Pinkstone says that overall nothing apparent came from the hours the Apple engineers spent troubleshooting at his dining room table, but he noted it’s an odd issue to recreate. He explained that there’s no pattern to the files that were lost. There were various file types and genres all lost with no relation to one another.
7 Comments RSS · Twitter
This is horrible since Music often gets confused over similar metadata. Think bootleg recordings of concerts or even older alternative versions of songs that were available on record or even CD but aren't in the iTunes library. I have quite a few bootleg recordings I transferred from YouTube into AAC files. I've not bothered looking through them all to see if any were damaged. Fortunately I kept a backup on a hard drive I put aside when I started Music last summer.
Also not clear is if the above is only true for Music only subscribers or if Music + Match subscribers are treated differently.
"Also not clear is if the above is only true for Music only subscribers or if Music + Match subscribers are treated differently."
I've been semi-closely following this for a while now, and the consensus Best Practices seem to be that if you care about your local music library, don't enable ANY iTunes online music services. All of them seem to cause troubles for folks.
And while file deletion is obviously the worst case scenario, there are other noted problems like metadata (tags) being changed willy-nilly.
(Obviously, frequent and rotating backups would save permanent damage. But still, it'd be a hassle to keep up with the rotations, and a hassle to do the restores...)
In a nod to your Claude Shannon post, that WAV files might have "10 times the number of samples" doesn't necessarily mean they'll sound better. :-)
I suspect somewhere inside Apple there's an engineer who both A. Knows what's going on, and B. Is politically savvy enough to stay far far away from this entire cluster.
Forgotten in the recent uproar is that Jim Dalrymple reported that Music ate his files in July of 2015.
@Jon That was my “Previously” link at the top.
"When Apple Music adds these matched songs to your iCloud Music Library, Apple Music doesn’t change or alter your original music files that reside in iTunes for Mac or PC or on your iOS devices from which they were added."
https://support.apple.com/en-us/HT204962