Friday, October 11, 2019 [Tweets] [Favorites]

Mail Data Loss in macOS 10.15

I’m working on more posts about the Catalina release, but I wanted to start with a short warning. I’ve heard a bunch of reports of data loss in Apple Mail. Thankfully, none seem to be caused by my apps. (Ironically, one of the bugs I’ve encountered is the inability to delete messages via AppleScript.) And, in fact, most of the damage has occurred without my Mail plug-in even being installed. Nevertheless, people contact me because it’s not unreasonable to wonder if third-party software is to blame, and I also hear from people who want a second opinion because what Apple support told them didn’t make sense.

What I’m hearing:

I don’t know whether these are due to Mail bugs or to other factors such as problems on the Mac or with the mail server. But my advice is to hold off on updating to Catalina for now. These sorts of issues are pernicious because:

  1. You may not notice that anything is wrong unless you are looking at the particular mailbox or messages that are affected.

  2. Because the data is synced to the server, problems can propagate to other Macs and iOS devices.

  3. Making a backup is difficult because, even if you set the preference, Mail no longer automatically fully downloads all messages. So the backup of the local data will necessarily be incomplete. (See EagleFiler’s Importing Attachments instructions for more about this. I’m happy to have most of my mail archived outside of Mail.)

  4. Restoring a backup is difficult because Mail data is constantly changing. There is no straightforward way to merge restored data in with messages received since the last backup, and also with the live data on the server.

Of course, it’s good to make backups anyway.

Apple advisors are apparently telling customers that if Mail data gets lost on Catalina, it can’t be recovered from a Time Machine backup that was made using Mojave. This didn’t make sense to me, and I’ve verified that it’s not the case. You can use Time Machine to get at previous versions of the folders in Mail’s data store, and then use the File ‣ Import Mailboxes… command to selectively import them into Catalina Mail. Since they import as new, local mailboxes, this shouldn’t affect messages that are on the server.

I also think that the advice to restore the whole Mac to Mojave makes no sense because as soon as you launch Mail it’s going to delete all the messages that were deleted on the server. In order to actually restore them, you have to make copies of the messages that might have already been deleted. That’s what Import Mailboxes does.

Update (2019-10-11): See also: Howard Oakley, TidBITS, Hacker News.

Update (2019-10-12): See also: MacRumors, iMore.

Update (2019-10-13): See also: AppleInsider.

Update (2019-10-14): Commenter Shiner says that the lost messages from “updating Mail’s data store” happens for local mailboxes every time Mail’s database is rebuilt, not just when migrating from Mojave, and reported this to Apple in August. It’s “trivial to reproduce” and lost 160,000 out of 200,000 messages in a test.

There’s some evidence that the moving messages issue is related to Microsoft Exchange accounts.

Peter Steinberger:

A doctor friend of mine lost about 20.000 emails when he upgraded to iOS 13. It was a POP3 account.

See also: TechRadar.

Update (2019-10-15): I’ve received four reports that messages moved to an iCloud mailbox (that were not originally in that iCloud account) simply disappear instead of showing up there. At first, we thought this was a Catalina issue, but it also seems to be happening with High Sierra now. My guess is that it’s an issue with certain of the iCloud servers.


Benedict Cohen

I experienced data loss when upgrading from iOS 12 to 13. I had some drafts that were not synced with the server. When I upgraded the tabs for the drafts remained but their contents was missing. I wonder if the Mail apps on iOS and MacOS share code.

@Benedict Yes, I forgot to mention that I also heard similar reports from users updating to iOS 13 (before Catalina was out). I don’t recall this happening with any prior OS update. I think they share a lot of code.

[…] Tsai, the developer of the long-standing SpamSieve plug-in for Mail on the Mac, has written a blog post warning about potential Mail-related data loss when upgrading to macOS 10.15 Catalina. In particular, he’s hearing from numerous customers that […]

Thanks for the warning — I’d be interested in hearing what strategies you use to archive your messages outside of mail.

@Morgan (1) I regularly archive messages to EagleFiler, which I wrote partially for this purpose. It converts Mail’s data store to standard mbox files and checksums them so their integrity can be verified later. (2) For messages that I don’t actually need to access across multiple devices, I have Mail rules that move them to local mailboxes (“On My Mac”). This ensures that there’s a complete local copy (with all the attachments) that will make it into the regular backups of my Mac. (This only works when the messages are processed by rules as they arrive. If you manually move messages to local mailboxes later, even if via rules, that does not force Mail to fully download them; it can actually lead to data loss as Mail saves the incomplete message locally and deletes the complete copy that’s on the server.)

The main issue I and others have noticed is that shows blank Sent folder/mailbox from GMail/GSuite accounts. Here is a thread on the issue from other users:

There's also Mail Archiver X, which can archive emails either directly from IMAP or from the local's database.

[…] Tsai, the developer of EagleFiler and the SpamSieve plug-in for Apple Mail on Mac, has written a blog post warning macOS users about potential data loss in Mail when upgrading to macOS Catalina 10.15.0 […]

I lost a lot of email messages when upgrading to Catalina. I would not have noticed except that some of my email accounts - which, apart from my Apple ID accounts - are all POP accounts so are not on any remote servers, seemed to have a surprisingly small number of messages after upgrading. I have a total of more than 100,000 messages going back about 15 years. On checking I found that some of my accounts were missing thousands of messages. Which was obviously very worrying.

Luckily I had backed up before upgrading and I painstakingly imported the missing emails from my backup. It took many hours and wasn't altogether successful because some sent emails are now logged not by recipient but by sender (me) which is not very useful. But I think/ hope I managed to find almost all the missing emails.

I am not a Mac expert but I noticed that after upgrading Catalina had created a new folder: user/library/mail/V7 which was not in Mojave, where my messages are now stored.

Perhaps this has something to do with the problem?

I should add that I had backed up not via Time Machine, which I have always found problematic, but via SuperDuper so I had a complete backup from which I could copy my missing messages.

[…] blog post by Michael Tsai via MacRumors is today suggesting that macOS Catalina seems to be experiencing a […]

Beatrix Willius

When the users move from v6 to v7 would the size of the folder be very different than before?

Import Mailboxes isn't a good idea because you would end up with even more duplicate emails than you have anyways in Mail.

The problems with deleting emails with AppleScript isn't really new. You can't even check if the deletion went okay or not.

There is a bug in Mail under iPadOS 13.1.2. If you send an email to yourself (say to read on a different device), it will appear in your Inbox and in your Sent box, as expected. If you move the copy in your Sent box to the Trash and then Delete it, the copy in your Inbox is deleted, too (and from your other device). That didn’t happen under iOS 12.4.

[…] blog post by Michael Tsai via MacRumors is today suggesting that macOS Catalina seems to be experiencing a […]

@Sam Each major new version of Mail migrates to a new folder, i.e. V6 on Mojave and V7 on Catalina. So that’s normal, and it’s good because it keeps the different versions of the data store separate.

SuperDuper is great, but I think it doesn’t fully support Catalina yet.

@Beatrix I think the file format hasn’t changed much this year (whereas a few years back the way attachments were stored changed). So in theory the sizes should be similar. However, I have also seen (going back a long time) that sometimes when the data store is rebuilt it throws out all the local copies of messages that are on the server. So in that case the V7 would temporarily be much smaller.

Yes, duplicates are a problem (#4), but that can be fixed. It’s much better to have duplicates than to lose your mail.

Deleting e-mails via AppleScript has always had issues, but it’s gone from working something like 99% of the time to failing 99% of the time.

Since updating to Catalina (and the new version of Spamsieve) my emails are no longer removed from my inbox, though they also appear in the Spam folder. I have a MacBook pro 2016 using Apple Mail, with Gmail, as the account/server. Any thoughts?

@Eric I haven’t heard of that issue before. Please contact SpamSieve support.

I routinely go through the pain of manually exporting all of my e-mail off onto .mbox to prevent this kind of thing, since ever since they started messing up .emlx and storing attachments separately that the SQLite metadata stores have had a number of issues.

I suspect that in many cases mail messages are indexed by something else than the Message-ID header (which is in itself prone to duplicates) and that the indexes get confused, but overall (and having used since the NeXT days), whatever has happened this time around is an(other) egregious failure of QA at Apple. There simply is no excuse for the amount of bugs that have shipped in macOS (and iOS), and for someone like me (with an e-mail folder tree that goes back to 1998) this means I won't be upgrading to Catalina until they sort it out.

(Ironically, I am more likely to move to an iPad Pro full-time for my personal stuff and set up a Windows or Linux box to work - I have a lot more trust on either of them to deliver stable releases right now...)

Not only data loss but the whole e mail program does not work on a mac book 2018. On a mac book pro early 2013 everything is almost ok. Sad because the mac book from 2018 is for work. Deleting te boxen en re install does not seem to work. disaster!

@Rui Unless it’s been fixed in 10.15, the built-in export from Mail to mbox is problematic because it doesn’t properly escape the “From ” lines (so message boundaries will get messed up if you import into anything other than Mail, which reads the boundaries from sidecar metadata) and it doesn’t download attachments that are not already cached locally. EagleFiler avoids this by generating the mbox itself and checking that all the attachments are there, and it also saves the metadata that mbox doesn’t support.

[…] dans cette même application. Il a expliqué la situation détaillée dans un article posté sur son blog : apparemment, un nombre non négligeable d’utilisateurs a rapporté qu’une mise à jour des […]

[…] a blog post by SpamSieve plug-in developer Michael Tsai, Mail is seen to be causing problems when users […]

Have you waited for one or two days after the upgrade? Have you tried re-indexing with Spotlight?
I had problems with my 60.000 items mailbox almost during every MacOS upgrade before; and the symptom was missing emails. In the end, I suspected it has to do with Spotlight: The list of mails in mail is created using a Spotlight query, and upgrading leads to re-indexing all mails in Spotlight - which takes time (and there is *no kind if indication from MacOS which is really a pain*). During that indexing, a lot of mails will simply be missing. In particular tough cases I launched re-indexing with Spotlight (forgot how to do this, but you‘ll find it easily) and that helped (and you can see whether it is finished by looking into Activity Monitor).
Can‘t say yet anything about Catalina, but it would be interesting to know if anyone who already dared to do this can solve it in this way?

@Peter I think Mail uses Spotlight for searching and for certain views like smart mailboxes, but it uses its SQLite database to browse normal mailboxes. So, if the messages are there, that should work without Spotlight.

@Michael yeah, I know about the "From" thing (I maintain, but I was under the impression mbox export would download attachments (since I have that option set to "always") even if they weren't extracted from the MIME info (or even downloaded already).

Can I take an EagleFiler archive and reconstitute a full IMAP tree from it?

@Rui It’s just nested folders of mbox files, so I don’t see why not.

I upgraded and then saw this thread! I have not opened the Mail program yet, and was just about to re-install Mojave. But before I do, I wonder if I use POP accounts (still) on my own company‘s server, whether I’m still at risk of losing emails.
Any clues?

@Sarah I have not heard of any problems with POP accounts, although that could simply be because most people don’t use POP these days. At least with POP you can make complete backups and don’t have to worry about corruption getting propagated to other devices. Commenter Sam has lost messages in POP accounts during migration. I’ve not heard of any POP messages lost moving them between mailboxes.

@Michael Tsai Thanks. I probably should go back to Mojave anyway, yes? I had another issue when I went to turn Time Machine back on after installing Catalina, I got a message saying that I had two Macintosh HDs and had to rename one . . . Have you seen this?

@Sarah Personally, without a strong reason to upgrade now, I would go back. You may even be able to do that easily/quickly by restoring the prior APFS snapshot by booting into Recovery. I’ve not seen that Time Machine error.

@MichealTsai @PeterResele From my experience with the innards of Mail (and I have a fair bit after working on plugins for 15 years), both regular mailboxes and spotlight mailboxes are populated by performing a spotlight search to find library ids (ie, the integer part of the .emlx filename) and then looking up those messages in internal sqlite database. Mail also uses spotlight for doing the message counts for mailboxes.

Given this, rebuilding spotlight indexes may resolve the missing messages issues for some but mileage may vary.

As an aside while I personally think this is inane because Mail has a highly reliable, performant sqlite database, I think Mail uses spotlight to prevent a discrepancy in which a person will do a system wide spotlight search for a message will turn up the same result as a search inside Mail, and a few other reasons.

@Scott I think we’ve both been writing plug-ins for Mail since before it was using SQLite and before Spotlight existed. :-) Yours certainly hook into it more deeply, though. It’s an intriguing theory. I don’t know what to say, because it doesn’t square with my experience successfully using Mail with Spotlight completely turned off, the fact that Mail’s SQLite database has fields and triggers to maintain message counts, or the fact that I’ve fixed messed up counts by editing the SQLite database. I’ve also wondered why they don’t they don’t use Spotlight only for the full content searches, given the performance of SQLite.

Over on the MacRumors forums, some people suspect that the data loss problem may be specific to Outlook accounts. Do you happen to know if the folks who reported the issue to you also have Outlook accounts?

[…] macOS Apple Mail 插件開發者 Michael Tsai 的網誌,他警告升級 macOS 10.15 Catalina 的用戶要小心備份電郵,因為從 Mojave 升級到 […]

Jörg Gudehus

I'm wondering that all the smart people here dont know how to use Time Machine properly. You have to be in Mail and start TimeMachine from there. Now you can restore single Mails or folders directly. No importing Mailboxes from the Library.
Please correct your post.

[…] El más reciente, reportado por Michael Tsai, desarrollador de EagleFiler y SpamSieve, un complemento para la app Mail de macOS, asegura que el gestor de correo electrónico por defecto de Apple puede provocar pérdida de información si se actualiza a macOS Catalina, según ha comunicado a través de su blog personal. […]

Victor Lazzarini

Just a report here for other users:

(1) I have not noted any loss of email, but I noticed today that one message did not have its attachment downloaded.
(2) Search stopped working -- it's really annoying. I have had to go to the outlook app to search for
emails, nothing is found on mac mail (it just shows that it is "searching"). I am currently rebuilding one of my mailboxes
to see if it gets fixed.

I updated a few days ago and things were mostly fine, Mail wise. Turned on this morning and all I have is Headers, no content in any email. Had old saved MailData, moved that in, no difference. It's all gone. I am lost for words.

[…] وحدد مايكل تساي Michael Tsai، الذي يطور إضافات EagleFiler و SpamSieve لتطبيق آبل للبريد الإلكتروني Apple Mail على نظام التشغيل MacOS، بعض المشكلات التي ظهرت في تدوينة. […]

[…] upgrading to macOS Catalina. If you depend on Mail for your work or in your personal life, please, read this before upgrading. […]

I have an open ticket on this going back to August (and was probably the first to raise the alarm on this one.)

Where you will see this issue is with "On My Mac" mail. If the Mail application launches with the "Mail Message Import" box (that you can't ignore), when Mail does it's "reimport" of all your mail, you will most likely have a number of "On My Mac" messages just flat out deleted. This is new behavior in 10.15 and wasn't taken seriously enough (IMO) when I reported the problem.

It's trivial to reproduce (if you have backups!) by deleting the "Envelope Index" files (Google around for how to do this -- it's been a long-standing recommended way to fix "search" issues *within* Mail), . It's not related to moving the Mail store from "V6" to "V7", and shouldn't (at least in any of my testing) affect IMAP mailboxes (as those just refresh by pulling mail back down from the server.)

I have test mailboxes with 200,000 .emlx files that end up with only 40,000 .emlx files in them after Mail does it's reimport. If you have "On My Mac" mail, you should test this yourself and immediately file feedback on it!

@Peter It does seem like Outlook/Exchange is overrepresented among the people with the moving messages problem. But there have also been multiple reports (for a different blank messages issue) with Gmail. And the importing issue seems to affect IMAP accounts and even local mailboxes.

@Jörg Have you tried using Time Machine from within Mail on macOS 10.15? Every user I’ve talked to said that this caused Mail to crash, as it did for me. It’s also not clear to me that Mail Time Machine can restore mailboxes that were backed up from previous versions of Mail.

@Shriner That’s really bad news if it’s related to rebuilding the Envelope Index database rather than just migration. There are so many problems that can only be fixed by rebuilding, which has historically been very safe. Even if IMAP messages can be pulled down from the server again, losing the local files in a rebuild would discard information that doesn’t sync to the IMAP server, such as colors.

[…] Developer Michael Tsai shares reports on users with Mail problems in macOS Catalina specifically missing messages or incomplete messages. In some cases you will see headers only and a blank message. Tsai recommends not updating to macOS Catalina yet, but if you already did he shares one way to fix this issue. […]

Yeah, I've never seen this in all my testing that it's related to migration *unless* the Migration forces the reimport (which, I believe, shouldn't happen intentionally.). I only noticed this because one migration I did forced this reimport and I didn't realize I had lost mail for a week or so after that -- the mailboxes (as seen in Mail) will still show the same number of messages -- it's only when you go to open them that you find they open blank (and then you can see the .emlx files are gone...)

My gut feeling is that people who have upgraded to 10.15 have already lost a lot of e-mail, but don't realize this yet.

[…] y SpamSieve, complementos para la ‘app’ Mail de macOS, ha difundido a través de su blog que ha recibido varios informes de usuarios de macOS Catalina que afirman haber sufrido una […]

Michael, This is Sam again. Just to confirm, further to your answer to Sarah about POP accounts, my lost messages were from POP accounts.

Hello all! Alse a simple POP3 account make Mail to crash. In my case I have multiple accounts (mostly with POP3) but the troubles come only with one that is very huge. Maybe the problem should happens with very huge mailboxes.

[…] part, elle contient quelques bugs très gênants, dont l’un d’eux rapportés par le blogger Michael Tsai, concernant les e-mails et plus spécifiquement, l’app […]

@Sam Thanks for the correction. I’ve updated my comment.

Michael, thanks for this, very helpful.

I'm not sure if you are aware of another even more serious mail issue that a number of us are hitting, which causes Mail to be completely unusable.

The error message you get is: "Mail cannot save information about your mailboxes because there isn't enough space in your home folder" ... the popup with this message has only one option, Quit. The popup happens some period of time after mail loads, sometimes almost immediately, sometimes after 1-2 hours.

This is an error that has appeared in the past with Mail but the fixes that worked in the past (removing index envelope data and the like) do not fix the issue this time, nor does reinstalling Catalina.

Here's the issue on Apple discussion forum:

Thanks for posting. I look forward to seeing if today’s update fixes things.

Same issue here. Just lost a couple hundred thousand mails, randomly, all over the place, from the last 20 years.

Am restoring, thanks to Mike Bombich, using Carbon Copy Cloner, from nightly backup, which I had first mirrored to yet another disk.

It's running since a night or so, and the mails are coming back, with their folder structure. Just take one by one the content of each subfolder in ~/Library/Mail, and import it using Mail's import function.

This problem went unnoticed for a day over here, since the mails appear to be there. It is not dependent on what type of mail server you had been using - I had been using all POP3, IMAP, Exchange over the years. The mails that were lost were all local to the machine, i.e., long time deleted from the (in many cases long time not longer existing) mail server.

So it is not just Online folders, it is everything, everywhere, with no apparent pattern.

Beware. And Apple - WTF?

[…] plug-in developer Michael Tsai has posted a warning on his blog to reveal that after installing macOS Catalina, some users could end up with missing or incomplete […]

[…] warning arrives by way of SpamSieve plug-in developer Michael Tsai. His post is based on feedback provided by customers, though the Mail problems aren’t related to […]

OK, here is an update on this topic:

I've first of all gone back to backup and restored all of them.

This morning, all ok.

This evening, WTF? Again skeleton mails from today.

I think I've found some (frightening) systematic:

Don't move a mail that has an attachment out of an online folder. It'll just kill the mail content. You can copy it into another online folder, even from like exchange to imap, but if you move it out, it seems to become a skeleton.

There's more inconsistency in that; sometimes it doesn't even perform the copy / move operation.

So, summary: DO NOT USE APPLE MAIL for the moment.

I have a 2018 Macbook Pro and I'm having search issues with the mail app: the query does not find all of the relevant emails. I've tried rebuilding and reindexing spotlight with no success. The only thing that I found that worked was to manually look for the email and view it. Only then, it would show up in the search query.

I transfer all my emails to local. Since upgrading to Catalina, all emails that have an attachment are not transferred properly to local (all you see is the sender and subject line) and lost. It does not damage emails without attachments. This is incredibly annoying and disruptive.

Anderson Robert

When I upgraded to Catalina I experienced a new problem now of not being able to email back to myself, which I do with many emails. They all bounce as a Permanent Error. I have complained to Apple for years that Mail was behind the times and had lots of bugs. Now we get this new set of problems. Search is just not a valid structure, it is hit and miss. Signatures are so awkward to set up. So much is not clear from the help menu. Things just do not work they way they say. Now with Catalina I see sliding lines of text and mail folder that I have no idea what they are to do. They could have put some info help boxes into this it seems.

And the date/time of the incoming email cannot be placed on the left side like I had it. It is locked on the right column. Nothing in View will give this option. I need it on the left side.

There are no ways to color icons either that might help users.

I have not noticed any lost emails with Catalina (may not have discovered it yet). I had this problem long ago with Mojave back in the spring and tried to alert Apple. I lost 15 years worth of emails when I migrated to my new MacBook Pro in January. The MBP had a lot of problem root software they had to find and take out. In the process I lost the emails when he set up a test user account and I then went back to my main user account. The tech told me I had put them on my Comcast ISP in the Mail mail boxes and they deleted them but I know I had them On My Mac mailbox using IMAP. They would not restore from Time machine either, so they told me.

Another problem has came up with the migration to the MBP, either Mail or Comcast has been deleting all my emails over one month old. At first I thought this was a preferences setting on the MBP, which had been put in by the migration but it even exists on Comast ISP with the MBP settings ok now. I called Comast and complained and they said they pulled all my old emails from an archive file and put back in my ISP mailbox but they all show as coming from me with my name, so that is not much of a help.

Long ago I found a big problem trying to delete incorrect emails from Previous Recipients. They just will not go away and keep showing up in the TO field while not in the archive list. AND for a long time Mail has been deleting email addresses of people I may not have mailed in a year or so. They are just gone, removed, I did not remove them.

This is a sorry application for such a pretty good computer company, imo. Now I am new to these forums and don't know how to get replies or other info...but if someone wanted to help me send an email to me at citizen How does one email Michael Tsai directly?


The problem I have experienced since updating is that emails that I know have been sent to me, and I can verify were sent, are not appearing in my inbox. Yet, if I check with the sender and know the Subject, I can find them when I search Mail. Effectively, the incoming emails are - apparently randomly - invisible unless I know they have been sent and I know how to search for them. They are in the inbox, in the right place, and they are visible upon a search, but disappear again when the mailbox is revisited. This is a disaster, and I understand from colleagues they have experienced similar issues. Furthermore, the whole Mail system is unaccountably incredibly slow now, every time I log into it.

Man, nothing to add, but yikes! Fingers crossed this gets worked out. Anyone remember when Microsoft bought Danger and then promptly destroyed all the cloud data of Sidekick users? This is kind of feeling a bit like that. On a scale of "okay, we can fix this" to "Holy goodness, yikes!"

My problem with the catalina upgrade on my 2018 macbook air is with the undo delete message option. It no longer works, and i actually use it a lot. Also, with some messages, but not all, the scroll down option does not work. Little annoying glitches! and I haven't seen the option of 10.15.1 yet. Also apple has not responded to my questions. I guess they have bigger problems than mine.

MANARA Riccardo

Would switching to another email client like Readdle Spark solve the mail problem?

Yes, I too have experienced similar frustrating issues with Mail in Catalina. One additional problem, I typically move old mail from my sent folder to an archive of mail in organized folders on icloud. So, as others have said copying mail to icloud is just failing like crazy usually (for me) with a timeout error. The mail that failed to copy to the icloud folder has gone from its original folder and now shows up in a new "On My Mac" recovered mail folder. But the date for all of the mail items in that recovered mail folder in column view is now today's day regardless of when it was sent!!! The date in the mailheader is the original correct date. So if I select date with the new sorting dropdown menu the newly moved items are sorted with todays sent mail.
I guess that I just cannot move any mail until Apple decides to fix this mess.

Hi all,
My upgrade to Catalina on my late 2015 iMac went reasonably smoothly, until I logged into the 2nd User Profile on the machine, and saw an error message about having to import email mailboxes. Didn't get to note down the message, however, a brief synopsis of the issue & resolution is:
Ran upgrade to Catalina on Sat Oct 19.
Emails from a POP3 mailbox were randomly missing content in the Mac Mail client, from a time period of Oct 18 back to April 2018.
By necessity, the emails had been deleted from the mail server - lack of provisioned space.
The mailbox had been in existence for about 12 months, but the last 7 months or so of email had only headers, no content.
The emails from the last couple of days were valid with headers & content.
Upon further investigation, and rebuilding the mailbox a couple of times, the emails without content disappeared entirely. I could verify this, as MS Outlook had been setup and had downloaded a copy of each mail item into its own mail store. However it did *not* contain all the sent emails, and was not gong to be solution.
Found the local mail store on the filesystem, went into a Time Machine backup point just prior to the upgrade being run. Restored the entire /Users//Library/Mail/V6 (Mojave) to a temporary directory in Catalina.
Imported *all* the mail into a new folder in Mail application. Was given the option to select a number of different folders, but chose bring all mail items in.
Found all the missing content, dragged & dropped into correct folders, including Sent emails for missing time period within the Mail app.
Seemed to have fixed the problem, however will require further checking over the next day or so.

I've been seeing "strange" Mail issues (okay, mostly using GMail backed mail accounts) even before I considered Mojave... that was on !0.12 & 10.13... where my laptop was still on 10.12 and my desktop on 10.13, I did on several occasions had to "rebuild" my mail database as I experienced (especially older) mails without their attachments or just empty.

Lately, after I've "upgraded" the desktop to new hardware + Mojave, I've been encountering (on both the laptop & the desktop) emails that are wrongly put in the same mail thread ;( some old non-related included in the same one ;(

Also discovered that the majority of my local mailboxes (hundreds of hierarchical mailboxes, >500,000 messages) had only partially made the conversion to the V7 mail store. There were missing messages and no body or attachments for most of the messages that were there. I had no problems with the four Gmail accounts syncing. I restored the V6 folder from right before the upgrade to 10.15 using Time Machine in the Finder, then imported the local mailbox folders into Mail piecemeal starting with the top level hierarchical folders. If you select a mailbox folder for import directly, you will also get the UUID folder names it contains, so put the mailbox in a folder and select the containing folder if you want to import individual mailboxes. Consolidation with old folders is going to be fun.

Some comments relative to the main post - no Exchange accounts were involved. There were none defined in Internet Accounts at the time of the upgrade.
Search wasn't working at all, but rebuilding the spotlight index didn't help in this case (there were missing message files - i.e. no underlying emlx files and attachments), and rebuilding the Envelope Index did make things worse (fewer messages listed and body no longer showing up for most messages that it had been visible in list preview before rebuild).

I had issues when upgrading to High Sierra with losing some old archived email, but the Catalina update really destroyed my email. 20 years of emails, archived in folders by year, and most of them are blank. Apple was unable to help. While I have backups, they are imperfect since the issues started with the last OS. I am ready to throw in the towel and setup all new email addresses.

These are the test I've done with results;

Like most users have reported, I too lost a huge amount of emails by updating from Mojave to Catalina. I had a solid TimeMachine backup so I simply formatted my drive then installed back Mojave and restore from TM. That worked perfectly and all my mail was back the way it should be. I have an extra MacBook Pro 13" so I decided to do some more testing. I first created a new folder on my Desktop and exported all my mailbox via the export function. Then I did a clean install of Catalina on my 13", open the mail app and imported all my mailbox via the import function. This allowed me to re-create a perfect copy of mac mail Mojave to mac mail Catalina. Then I made a brand new TimeMachine backup of my 13" Catalina install. Now I'm thinking I have a TimeMachine Catalina backup with all my mail so that should be perfect! Well no it was not. I erase my 13" and proceed to install a clean copy of Catalina, then use my perfect TimeMachine backup to restore thinking that my mail should be back the way i left it. Believe it or not, upon restart and opening the mail app, most of my mail only show headers and nothing in the message. This is a terrible bug and looking at the release notes of up-coming updates, it looks like Apple is not even addressing this issue! This is terrible really! Bad Apple :(

Did the last update fix any of these issues?

@Chris No. I will update the post if I hear of any fixes.

[…] people do not update there Mac to macOS Catalina. If they are upgraded again he put a solution on how to fix the Mail problem. However, you can use the time machine to get you into the previous mail version. Again if the time […]

Miloš Mitrović

Is there any hint how catalina mbox files can be imported to Mojave? I have 120GB of mbox on Catalina mail, went back to Mojave, but all mail mbox files have same problem "no valid mbox files were found" when I try to import to Mojave mail app.

@Miloš Despite the “.mbox” in the folder names, Apple Mail’s data store is not in mbox format. So if you are using the Import Mailboxes command, you should tell it to “Import data from” “Apple Mail” rather than “Files in mbox format.”

I have lost all the content of an outbox mail of one of my gmail account when upgrading to Catalina.
I was in contact with Apple international support via telephone and they have confirmed me that this issue is known to them as a bug in Catalina, which will be fixed in the future.
I suppose that more and more applying to them would accelerate probably their speed.

Miloš Mitrović

@Michael Tsai Of course I tried "apple mail" option too but lots of errors in console with newer ending story of import. What about "Mail Library cannot be read because its format requires a newer version of macOS." is there solution to convince Mojave version to read Mail folder which was already in use by Catalina?
I am trying to solve 120GB emails which are present in Mail folder but not shown in Catalina, so I moved to Mojave and trying to get those emails in app...

@Miloš I had thought Mojave Mail would be able to import from Catalina Mail directly. If not, you could use my EagleFiler app to convert the files. Make a new EagleFiler library and then drag and drop all the Catalina account (UUID) folders inside the V7 folder into the EagleFiler window to import the whole mailbox hierarchy. This will convert all the mailboxes to mbox format. Then you can tell Mojave Mail to import the mbox files stored in the EagleFiler library.

I updated to Catalina yesterday. All of the folders for my iCloud email have gone missing. I also have a gmail account that appears to be working normally. Unfortunately, the iCloud account was being temporarily used for all of my business launch. My iPhone still has all of the folders and e-mail.

is the issue gone with 10.15.1 ?

Yes, would be great to get confirmation from Micheal on whether this is fixed or not with 10.15.1? Thanks

@abc @dfe I’ve heard that some Mail bugs are fixed in macOS 10.15.1, but not the ones mentioned in this post.

Mails for a whole year were wiped out and every time I try using time machine to retrieve them Mail crashes. There seems to be no way of getting them back and this is very disheartening.

I did a clean install of Catalina 10.15.1 today (Nov 1) and the bug is still not fixed.
All of my inbox messages (POP mail) from the past two years vanished sometime after I did the install. And for others, message headers are showing "no recipient" "no subject" and content is blank.

Would a clean install and a fresh sync of email from the server avoid these mail problems? Instead of trying to upgrade an existing email setup? Thanks.

The problem persists in 10.15.1. Almost 50% of my old emails have now disappeared (40Gb of info). Most of the emails only show headers and nothing in the message. When rebuilding the mailboxes, the emails just evaporate. I've looked into the finder structure and the emails are completely gone. I can only recover from backups, but many emails will be gone forever. Bad.

How can Apple release this with such a bug? It means you're taking a risk updating to Catalina, and one more serious than usual, because the data loss can propagate to your other devices.

It all seems odd - all that marketing for the new and great macOS, yet Apple must surely know that 'upgraders' are risking losing data! Unbelievable!

Might this blog thread make a note if and when the problem is resolved?

It is ridiculous that Apple would allow this to go ob for nearly a month. We are losing all corporate emails that have attachments (85%) as the email are blank once pulled onto our servers and personal computers. We do not store email on our server, they store locally and this a significant issue for our business.

We have worked on every possible fix and there is NO WORK AROUND to recover blank emails nor to stop the emails from converting to blank documents.

downloaded Catalina two days ago, went to check email and a blank email page opens without any listing of any emails. Mailboxes shows the various mailboxes but there is no information or display when you try them. Not sure where to turn for any fixes. My email is cloud based. Luckily email still shows on iPad mini and iPhone X S.

I think you guys have just helped me solve the big issue I have been trying to fix for the last three days. I upgraded to Catalina last week and have since lost 3 very important drafts in my Apple mail...they show up as drafts (which they are) and I can see the the preview with the first three lines of text in the drafts folder on my iphone Apple Mail App but when I open them, they are blank. None of these drafts show up in Apple Mail on my Mac, nor in Gmail. They simply never synced it appears. Sadly, one of the drafts was SUPER important to me, and now it appears corrupted and unsalveagable. I spent an hour on the phone with both Apple and Google support but now I think I understand the problem. I will hold on to the draft in in the draft box and hope that once there is a bug fix it comes back to life, but I am losing hope. I really need that draft. Any advice?

I experience the same problem. When I move or copy emails received from my outlook exchange account in another folder in Apple Mail, they appear to be totally empty, despite the fact that the sender, subject and date/hour appear correctly. You have to open the mails to realize the problem. The problem has appeared since the upgrade to Catalina. Other accounts (Gmail) are not affected by this problem. Only mails with attachments are affected. Other mails can be moved or copied without problem. Another problem is the non-synchronisation of the outlook exchange account. Mails read or deleted in Apple Mail do not change status on the Exchange server. As a consequence they appear to be unread or undeleted on my i-phone. This second problem seem to affect all emails in the outlook exchange account, with or without attachement. My solution (well not a real one, more a safety trick) as of today is to keep all mails from my outlook account in the inbox and postpone the cleaning until the bug fix is released.

This also happens with POP accounts in Apple Mail in High Sierra (10.13)(I don't use iCloud, so it's not just an iCloud problem with High Sierra). I first noticed it this summer: I couldn't find a message that I was fairly sure I had saved (all my saved messages are in a folder named Archive). A few weeks later, I looked for a different message that I KNEW I saved and that wasn't there either. When I looked at my Archive folder, it only had about 600 messages; there should be 2,000 messages. Restoring from Time Machine backups didn't work (every time I imported the TM mailboxes into Mail, the number of messages didn't change). The messages started disappearing around June 2019 (this is when the message counts in the TM mailboxes started going down).

What I finally did was use EagleFiler and imported every mailbox from every TM backup I had (yes, it took a lot of time, but I was able to restore every message, since I had TM backups going back to April). Now, I export my "Saved Messages" folder from Apple Mail to a .mbox file every few weeks and add it to my EagleFiler library, so I don't have to worry about losing messages.

THANK YOU, Michael, for creating EagleFiler, you saved my bacon.

If you really need to move a mail from the Inbox (from an Excchange account) to another Apple mail folder, you can copy the message on the Desktop, then open the message from the Desktop and copy it to the target folder. Doing it, the message is copied correctly. Do not forget to delete the message from the Inbox if you wanted to move it and not just to copy it. Obviously, such a procedure is far from being user friendly. At the very least, it shows to the Apple developers a step-by-step procedure that does not transform Exchange messages into blank messages. I leave it to them to make it automatic;-)

macOS 10.15.2 Beta 1 is out and Apple Mail is Version Mail 13.0 (3608.2). macOS 10.15.1 had Version 13.0 (3601.0.10). I don't know if anything has been fixed since I don't have any Mails in my Beta-VM but I wanted you to know.

Question: when does the data store upgrade happen - on installation of Catalina or on initial launch of Mail app? I installed Catalina a few weeks ago but happened to read this thread before opening Mail - wondering if the damage is already done, or if I can wait to open Mail once Apple issues a fix. (reading mail via iOS, fyi) Thanks!

@Tim On launching Mail.

[…] Mail trên macOS, đã phác thảo một số vấn đề đã được khắc phục trong một bài viết trên blog – ông khuyên nên giữ bản nâng cấp Catalina, nếu bạn dựa vào Apple Mail, cho […]

I made a clean install of Catalina (booting from usb stick containing 10.15.1) deleting Macintosh HD with disk-utility, then installing the operating system) and recovered my profiles via Time Machine. Locally stored emails and emails in pop accounts appeared blank except for the email header. I tried this several times. Then I made a clean install of Mojave and recovered my profile from the same time machine back up, and everything is fine. In other words, the problem is not a potentially corrupted back-up, but OS X 10.15.1
Each process about six hours, so I’m leaving it until fall 2020. CONGRATS APPLE!

Michael do you know if Apple is officially aware of the problem?

@John I’ve heard second-hand that they are. They haven’t responded to my bug report.

@downgrader What kind of mail accounts were you using that you were able to recover mail? I am looking specifically at the issue with Exchange mail.

Hi All,
I have been spending about 9 hours with an "APPLE expert" by the customer care service in Italy to try diagnose and fix the problems induced by Catalina on the mail client. No way, after every iteration it got worse. By now I feel like saying it looks very much related to the interaction with Exchange servers. Something got really screwed up. Back to the old story between Microsoft and Apple. I reported to the "expert" this is really something that should not happen; it makes no sense to release a new operating system without these very basic tests. I'm really disappointed.

@Massimo Was it an upgrade or a flattened install? So far it looks like it's all Catalina upgrades?

MANARA Riccardo

Michael, if I upgrade to Catalina and I don't have any interaction with Exchange Servers but only Gmail and Fastweb (an Italian server) accounts, do you think I could have problems?
Another question: if I no more use Mail but another mal client (i.e. Spark), could I avoid mail problems upgrading to Catalina?
Thank you.

Just to keep everyone informed: macOS 10.15.2 Beta 2 comes with a slightly fixed version of Mail.

10.15.1: Mail 13.0 (3601.0.10)
10.15.2 Beta 1: Mail 13.0 (3608.2)
10.15.2 Beta 2: Mail 13.0 (3608.

I don't know if anything has been fixed since I don't have any Mails in my Beta-VM but I wanted you to know.

@MANARA Yes, I have seen people have problems who are not using Exchange. Using another mail client would avoid them, however.

I can neither confirm nor deny that the "envelope index rebuild mail deletion bug" is *NOT FIXED* in any of the 10.15.2 betas yet.

So I can not confirm or deny that you *shouldn't expect it to be fixed in 10.15.2 at all* at this point because Apple said that a "lack of duplicate reports" is not moving this higher up the regression fix priority list.

I can neither confirm nor deny that we are not pleased about this by any means.

I *can* say -- that if you have tested this and you have reproduced the problem and you have not filed an official bug report about your results -- this is why it's not fixed yet. :-(

@Shriner Why didn’t they name Catalina GLOMAR?

Shriner, have you filed and official bug report yourself?

How will we know when upgrading to Catalina is safe? Has apple admitted the bug and given a way to track whne it will be fixed?

@Michael Apple has not publicly acknowledged it, but apparently they have replied to Shriner’s bug report. I will update this post if I hear that the bugs are fixed.

John Hart -- of course. During the 10.15.0 beta process (mid betas) we discovered this and filed it. We also pay for AppleCare Technical Services and filed it there, too. This is why we are most unhappy.

The only thing we haven't done is wrote Tim Cook directly. We are considering that. :-/

Lisa Greenfield

Im wondering if anyone can help me ...since upgrading to catalina , any time i begin to compose a message using mac mail but using my out bound webmail address , i receive this message below, it freezes my mail and I have to force quit.... the actual email server works when I log in through the website and sends the email zero

The message “Re: ” could not be moved to the mailbox “WEBSITE Email” ...

Mail was unable to open this mailbox on the server “”.

The server returned the error: Mailbox does not exist, or must be subscribed to.

@Lisa Greenfield

At this point I would not trust anything Mac mail using Catalina. If you're able to revert to Mojave without any lost of data, that be the road I would take. It's shameful what Apple is doing here. Tempering with precious emails in that way is clearly unacceptable. It's almost impossible that Apple did not catch that issue while they were beta testing Catalina! Yet, they do not seemed to care!

I wonder just how widespread these issues might be given Apple's response to Shriner. I migrated two 2016 15" MacBook Pros to two new 2019 16"MacBook Pros that had Catalina 15.1 preinstalled last weekend without any apparent Apple Mail related issues. It's been 3-4 days since I did the migration to the new Macs and have done routine spot checks of my mail & attachments without finding any issues. I DID import all current mail (~26,000 emails on one machine) into EagleFiler, after confirming download of all attachments, as a precaution.

Note that neither new Mac did a mail "reimport" to my knowledge and I have not attempted to rebuild envelope indexes. I have three imap accounts (Gmail, iCloud, and one associated with a personal website) although the vast majority of mail is in the iCloud account. I also have an extensive "on my mac" folder hierarchy. I do not have an exchange account or utilize Outlook. So far, so good but I'm still spot checking and hanging on to the EagleFiler databases!

I'm fed up with this. I'm importing all my mails in EagleFiler. Hey, how can I change the font size in EagleFiler's Mails and Lists?

The way I'm importing is to work through a local dovecot that I've dockerized.

@eMaX There are settings for the various fonts in EagleFiler’s Preferences window.

Sören Nils Kuklau

It’s almost impossible that Apple did not catch that issue while they were beta testing Catalina! Yet, they do not seemed to care!

For whatever it’s worth, I’m not running into the issue.

My guess is this particular issue was non-trivial for Apple to reproduce at the time, and didn’t see quite enough users affected to make that high-priority enough to delay the entire release. (Which, it seems, wouldn’t have helped anyway, as they still haven’t been able to fix it by 10.15.2.)


The problems seem not to be limited to the Upgrade. I still receive Emails with 10.15.1., can read them when in Inbox but when moved to a Folder and open it again, the content is missing. :-(

That can cause serious problems when the email is needed later for proof.


@Frank That’s correct. One problem is the missing messages during the upgrade or when the database needs to be rebuilt. But messages can also be lost or blanked when moving between mailboxes (either manually or via rules).


Disabling Mail Preferences --> Rules appears to solve the issue of receiving seemingly random empty messages into my Exchange accounts (IMAP), which I had migrated from an earlier version of OS X.

Hope this is useful.


I'm also hitting the problem with 10.15.1 and corporate Exchange email account.
I'm observing following problems:
- when I copy messages from Exchange Inbox to a local folder, the messages with attachments are not copied properly, only header is seen. It seems reliably reproducible. Copying messages with attachments from an IMAP Inbox works OK apparently.
- when message is moved from Exchange Inbox to a local folder by an Inbox rule, even messages without attachments are not moved properly.
- is constantly showing "Moving Messages" with the numbers slowly jumping up. No idea what is it doing, and restarting Mail doesn't help, it starts counting from zero again.

I've contacted AppleCare, showed them the problem, they collected info from my machine, and now I'm awaiting a reply.

I also encountered problems with email after upgrading to Catalina. On the phone with Apple for hours, the guy telling me they'd had a lot of problems with this update, and they hadn't really tested it properly. I rolled back to Mojave. Now wondering whether they are ever going to address the issue. Any recommendations for an email client I can use instead of Mac Mail, where I can import all my emails and settings, anyone?

Yes, I can confirm that after weeks, I still experience *serious* Mail issues (made the mistake of installing Catalina from Day One and then hoped for updates, all installed). It's now "better" than at the beginning, but still totally bad.

In addition to the points mentioned previously (pasted below), I also have to manually delete everything in (iPhone), so any spam I delete in Mail (MacBook), I must delete twice there plus once in the app…

from Frank:
"The problems seem not to be limited to the Upgrade. I still receive Emails with 10.15.1., can read them when in Inbox but when moved to a Folder and open it again, the content is missing."

from Robert H:
"I'm also hitting the problem with 10.15.1 and corporate Exchange email account.
I'm observing following problems:
- when I copy messages from Exchange Inbox to a local folder, the messages with attachments are not copied properly, only header is seen. It seems reliably reproducible. Copying messages with attachments from an IMAP Inbox works OK apparently.
- when message is moved from Exchange Inbox to a local folder by an Inbox rule, even messages without attachments are not moved properly.
- is constantly showing "Moving Messages" with the numbers slowly jumping up. No idea what is it doing, and restarting Mail doesn't help, it starts counting from zero again."

Is there any indication that MacOS Catalina 10.15.2 Beta 3 will address the critical problem of lost emails bug(s)?
Like others I will finally migrate my 'archive' of emails to FileEagle librarys once I see the path to reconstructing what Catalina has destroyed.

First headway in ages: I saved my Mail folder (Library) and imported it in Mail on a different Macbook -- eh voilà, everything there, with folder structures AND attachments!


Can you be more specific? Did you get Catalina to work? or you copied the entire mail (V6) library and import to another mac NOT running Catalina?

@ John Hart: From Catalina to Catalina. But the from-Mail was completely screwed (see my first comment above). I went from my old MacBook Pro which was already on its last legs, had Catalina, and experienced all system updates for the past 6 years to the Library folder (Finder: press Option > Go > Library) and simply copied the Mail folder on an external drive. I then went to my brand-new MacBook Pro which I set up from scratch (NO migration), opened there, and imported the Mail folder from the drive (File > Import Mailboxes: Apple Mail). Hey presto, took a while (220K emails) but it looks nice now. :-)

Kakis so there is no need for exporting Mailboxes in .mbox format? Will this technique work when going from Mojave to Catalina? I used to only copy Mail Library folder on older systems, but it stopped working few OS systems back. Are you saying I can copy my Library folder to external HD and import this very folder in Catalina and all my mailboxes, rules , etc will be copied?

Just got bit. After upgrading, all was fine. Then I tried fixing an issue with a smart mailbox displaying that there were messages in the mailbox when it was empty. Boom, hosed. My accounts are all POP (except iCloud and Gmail, which I don't really use but we're all forced to have). As others have said, only the large boxes are affected. I can't say how large they need to be, but my boxes with Amazon receipts, my wife's emails (going back to when we first met in 2002), and base inboxes full of unfiled messages are toast. If I go into Time Machine, they show up, but I'm not able to restore -- even from a TM disk I just started a few days ago. Diving into both the current V7 folder and ones from TM backups only nets me partial success. So far, the most recent message I can find in there are from three weeks ago. Both in TM and on Crashplan. Where is Mail storing more recent messages in the meantime? They're there somewhere. Also, new message since this happened are fine. So far.

@ Kakis: Did Mail rebuild the Envelope Index database? I can also copy my Library folder (V7) from my second, older Macbook and it looks and works fine. However, I'm not able to perform a single search. And as soon as I rebuild the database, all the data of folders containing +2k emails gets lost.

Can anyone else confirm a cutoff at around 2000 emails per folder?

I found a workaround for my system! After having bought a 2019 16" model last week, I upgraded my 2014 Macbook Pro from Mojave to Catalina and migrated all data to the 19" model. Mail worked fine for the 2014 model and was simultaneously heavily screwed up for the 2019 model as mentioned above - perhaps the one big difference being that after migration the latter entirely rebuilt its Mail database, whereas my old Macbook apparently didn't. I tested that a few times, deleting the current V7 folder (which would go from 6GB to "Import Mailboxes", and then selecting "Apple Mail" rather than only chosing particular mailboxes. After importing the intact V7 folder from my older Macbook this way, all messages showed up correctly and were searchable. The import does jumble up the order of folders and mailboxes, but that can easily be sorted out manually. I then imported signatures and rules and have a working Mail App without any data loss so far. Hope it stays that way and hope this is helpful for someone out there!

And btw, just to be safe I installed EagleFiler which does an excellent job in archiving all or any mailboxes and folders.

I don't know what to say other than what I wrote before (I'm not a computer person). As I said, I had thousands of emails, some folders also had more than 2k (to answer one query above). All I did was copy the Mail folder in my library -- but at that time, the old laptop was already in Catalina (so there weren't any V7 folders any longer). The main thing is that when I updated my old MacBook Pro to Catalina, all hell broke loose and Mail was rubbish with all attachments gone/not displaying -- and when I coped the Mail folder from that faulty laptop onto the new MacBook Pro (clean, no migration but with Catalina), everything was fine.

@Bernd -- there's no rhyme or reason about what mailboxes will have deleted mail as far as I can tell. The mailbox I sent to Apple has 1000 emlx files in it, but only 7 after the Envelope Index-deletion-reimport.

10.15.2 is out. Any news on the subject?

@Tasos: I'd love to know this too. Does 10.15.2 fix the issue?

10.15.2 does *NOT* fix this. If you have not filed this bug with Apple, what are you waiting for?

@Shriner, I am having the same issue with all of my POP and IMAP accounts except for my iCloud account and I have filed FB7490433 with Apple. I don't use gmail, just POP and IMAP mailboxes from my hosting provider.

I note that prior to my Catalina migration my Mail folder had 5,792,038,039 bytes (5.82 GB on disk) for 101,388 items and after running the Mail app after migration I had 2,871,906,047 bytes (2.8 GB on disk) for 37,633 items in that folder. All of the message envelope meta data appears to be there, but as I go back in time a few weeks on some folders, a few months on others, the message bodies are all blank.

I was able to import all of the mailboxes from the good backup into AirMail 3 (and into Mail Steward as backup) and all of the data was there.

I’m posting just to underscore a point that Shriner made above but that I don’t think has really been emphasized enough (both here and elsewhere with this issue): the issue ONLY affects local “on my Mac” mailboxes.

After careful preparation – 3 separate backups, and pruning and archiving (to mbox files) of my mail – I just upgraded to Catalina 10.15.2 yesterday. I was somewhat surprised when I crossed my fingers, held my breath, and launched Mail fo the first time....

It just launched. There was absolutely no updating or conversion of any of my mail. It was no different than launching Mail on Mojave.

The key which I see now, but wasn’t clear to me then: I only have IMAP (and Exchange) mail folders, with nothing stored locally on my Mac.

I realize for many people this potential bug is still a deal-breaker,but for anyone out there who I has been waiting to upgrade and either (A) like me, only has IMAP mail, or (B) can move things around so that they only have IMAP mail – then for those folks there is no reason not to upgrade now.

@Sam There are multiple issues. The one about upgrading and rebuilding mailboxes seems to only apply to On My Mac mailboxes. However, other users are having IMAP and Exchange messages disappear when moving them between mailboxes (even on 10.15.2). This doesn't seem to affect most users, but since we don’t know who will be affected I can’t recommend updating in general.

@Michael. Thanks. What you say makes sense.

It does make me wonder, though. It sounds like the "on my mac" problem has a clear source in Catalina: rebuilding the mail data store and/or the envelope index in Catalina Mail leads to corruption and a loss of messages. Shriner has found a reproducible bug here and reported it to Apple.

The IMAP and exchange problems don't have a clear source or explanation, and I wonder if they are even caused directly by Catalina. I know I've had Exchange problems in the past and they were almost always caused by a conflict between the huge variety of Exchange servers out there (including very old versions), and the interaction with them by the mail client.

Still, I'm going to watch my IMAP and Exchange folders carefully. I routinely move messages from an Exchange Inbox to a series of IMAP folders, so if the moving message bug is alive, I may still be exposed to it.

@Sam I think it’s Catalina because some of the people who reported these issues to me reverted to Mojave and the problem went away.

Sherman Wilcox

This is probably and edge-case question, but I only use IMAP with Fastmail. (My university uses Exchange, but I forward everything to Fastmail.) Does anyone know if the problem appears with IMAP on Fastmail?

The "On My Mac" Mail deletion bug? Still not fixed in 10.15.3 b1. We are besides ourselves here about this. Grr.

Sherman Wilcox

No, I rarely use "On My Mac. I store everything in Fastmail folders. I thought I read above that the deletion bug also appears with IMAP.

@Sherman The “moving a message to another mailbox instead deletes it” bug can happen with IMAP. I don’t know whether it’s limited to certain servers. I have definitely seen it with iCloud and Gmail. Not sure about FastMail.

With the update to 10.15.2 in Apple Mail I lost about 400.000 mails, which are still displayed with subject, sender and recipient but have no content anymore. Both in the preview and after opening the messages are "empty".

I noticed the whole thing after certain messages, which I normally access via search, were no longer displayed.

I then re-indexed the mailboxes via Onyx without success. Even more messages seem to have completely disappeared from the mailbox folders.

A recovery of the Library/Mail/V7 folder from a TimeMachine backup from 2 weeks ago was unsuccessful because 400,000 elements from the backup were "copied" back into the folder and Apple Mail subsequently indexed the mailboxes (also about 400,000 messages), but these are still all empty in the mail.

Does anyone also have this problem and a solution?

@Patzi. If you have a successful back up of your "on my mac" mail -- restore it to a different location (like your desktop). You should then be able to successful *import* the mail (File --> Import Mailboxes).

That works.

But please file a bug with Apple.

Great advice everybody - thank you!!

This is my issue:

I bought a new MacBook Pro 16". I use Apple Mail consisting of an iCloud address and 2 gmail accounts. I have close to 1.5 million e-mails. I had issues migrating from my older MacBook Pro 15" to this new 16". The older 15" used Mojave, the new one, Catalina. About 1/3 of my e-mails did not migrate to the new computer and many that did, only show headers - no body. Apple customer service really has no good answers. One idea was to upgrade the 15" to Catalina, then try to migrate. But I did try that before and after upgrading to Catalina, Mail froze during the set-up (kept on trying to download e-mail). It was fixed by reindexing and going back to Mojave, so I never got far enough to try a migration from a Catalina 15" to the Catalina 16".

Do you recommend reverting to Mojave on my 16" or trying some way to import the e-mails from the 15" via Time Capsule, external HD, a 3rd party box import program, etc.?

Thank you for bringing this critical information to the forum.

This really sucks. In my case I could still reproduce the problem under Catalina 10.15.2. Every time the reindixing is done more than half of my messages disappear or are only available without body.

@Shriner: thanks for the recommendation. I decided to wait with reimporting until this problem has been fixed by Apple.

Missing mail after Catalina update. Nearly 90,000 historic emails were lost, in my case, reducing the Mail folder size from 35.5GB to 2.7GB. Fortunately I have found an old backup with much of the historic messages. However, key messages remain missing including copies of customer invoices sent out by email. Destroying people's email archives has the potential to put companies, especially smaller folk, out of business. The problem is only obvious once one starts actively looking for those old messages. But, by then, it may be too late ... old backups have been replaced by new backups that lack the older messages. Even if one can recover messages from a backup, the time and effort ( and $$ ) involved in doing so is significant. Shame on you Apple. Protecting user's files/data has to be the #1 objective in any OS update.

How did we find out: we are missing our records of emails (the messages) sent out via Apple Mail!

Oops, emailed customer invoices.

Again -- if you have not *personally filed a bug report* -- you are not going to get this problem fixed any faster.

Apple needs to get more bug reports in on this ASAP.

If you use Time Machine you can recover your lost mail. Apple replaced the V6 folder with a V7 folder in Library/Mail. I have no idea how they screwed it up. Doesn't really matter at this point. We just want our mail back.

Open Time Machine and look in Library/Mail for the last backup with the V6 file. Restore it. This will put a copy in your current Library/Mail folder.

Take it out and put it on your desktop. Open Mail and select menu item "File/Import Mailboxes". Choose the V6 file on the desktop. This will put all of your mail up to that date in a folder called "import" in the Mail sidebar. You can drag and drop to selectively replace your lost mail. A bit tedious but better than losing everything.

Happened to me too. Drag and drop was losing attachments. I tried rebuild. Then it said it had to rebuild the index and import everything. 45 minutes later, my emails are massively corrupted. Lots with header only, no body. Many missing. Bunch of random stuff in "Recovered", mostly stuff that I had deleted. Worst problem with my Mac that I can remember in 20 years. Apple, please get on the ball!!

@Sherman — it was interesting to see your reference to FastMail, since that's what I use too. I'm actually delaying purchasing a new Mac because of this, since I'm doing alright on Mojave on my existing Mac, but I'm sure the new one will force a Catalina move. Since I only have FastMail and iCloud email, I'm thinking to just deconfigure iCloud email and go to a single IMAP account, which it appears from this thread shouldn't be a problem running on a single mailbox. I'm also thinking that if I really want to keep both active, maybe I should move to Outlook 2019 or 365, or to Thunderbird, or the FastMail web interface (which isn't great but is good enough), and get out of this quicksand entirely (since all I need to do at most is re-cache my email). It's insane that this is still an active problem by 10.15.3b1.

@dpz I have used both Postbox and MailMate (currently MailMate) since Catalina and have no problems. (FWIW, mileage varies on the Fastmail web interface. I rarely use it but when I do I find it fast and uncluttered.)

Just opened a "feedback" on the IMAP-move-to-other-account-issue. Feedback-Assistent lists "no similar feedback" :(
Will reproduce the on-my-mac-issue soon an open a feedback for this too.

Sören Nils Kuklau

> Feedback-Assistent lists "no similar feedback" :(

I wonder how "similar feedback" works. Could be a manual process, could be a very primitive keyword-based algorithm, could be machine learning. Either way, probably a lot of false positives/negatives.

I updated from Mojave to Catalina beta a while back and my mail was fine.
When I got a new Mac and reinstalled from a TM Backup via recovery the mailboxes where reimported and that is when I lost about 10.000 mail that were in a local mailbox.
After trying many things I installed a TM backup from before Catalina and grabbed the mailbox file of the local mailbox from a TM backup from the old mac running on Catalina.
I added this to the library of my reinstalled Mojave, deleted the envelope files and after reimport in Mojave I got my mails back from my TM backup that was run unter Catalina.

I will strick to Mojave until this is sorted out.


I just had a thought that I could run Mojave in a VMware Fusion VM for my Apple Mail until they fix their issues. Doubt I'll bother doing that, though.

Although my issue has nothing to do with Catalina, it does touch on the Time Machine
Mailbox recovery strategies mentioned in this thread. My issue was that I deleted a good
number of emails from my IMAP Inbox before archiving them, and then had to recover them
from Time Machine. At the time of the backup, I was running High Sierra on my machine Rockweed.
Now I am running Mojave. (I have learned to be very conservative in moving to new versions of OSX)
- recover V5 folder from Icarus, which contains my mail from last august:
copy to cernboxKellogg/Mail_recovery_2019-aug (note that files in cernboxKellogg are mirrored via the cloud
and available on all my machines)
- with Mail, Import to Import-2 on Rockweed as Apple Mail (since import as mbox claims it can find no mbox)
but when I try to move messages to another local mailbox (2017), I fail, with crazy message that it could not find
Mailbox "Happy New Year from Amherst", which is the title of one of the emails I am trying to move
notice that the mailbox structure is complex, and also that this backup was made under High Sierra, not Mojave
- try the import on maccmskel4, which is running Sierra
for INBOX-mac.mbox I can import it without problems, and I can move all the mails without attachments to 2017
I can view and download the attachments, but cannot move them, getting the warning that the body cannot be loaded
For INBOX-cern.mbox the behavior is exactly as on rockweed: I cannot move any email after importing it
getting the warning that the body cannot be loaded
rebuilding INBOX-cern does not help
- conclusion at this point: OSX versions are not the primary issue (even tried on ElCapitan with same results)
- EagleFiler import succeeds with copy on cernboxKellogg: cernboxKellogg/Mail_recovery_2019-aug/INBOX-cern.mbox (18300 records)
in the EagleFiler window, I see the emails listed very much like in Mail
the # column gives the number of attachments - if I click on the attachment, the entire message (including attachments)
opens in Apple Mail
- Since the Eagle_Library/Files/INBOX-cern flat file obviously contains all the attachments, in-line,
try importing it using Mail/Import as mBox to Import-4
get 18175 messages and a warning that some could not be imported - there are no attachments
Now I get somewhere:
- rename Import-2/INBOX-cern.mbox to 2017-cern and then drag 2017-cern to top level of On my Mac Mailboxes - success!
all the attachments remain visible (this procedure works on all my machines)
- I now delete all the non-2017 mails, and repeat the entire procedure for INBOX-mac and for 2018 and 2019
- only slight issue remaining is that I am not able to find a way of merging the -cern and -mac folders


- something poisons the structure of the Mailboxes recovered from Time Machine
in such a way that neither EagleFiler nor Mail can handle them properly,
particularly their attachments
- the mail and attachments are recoverable, but their Mailboxes are extremely fragile
Basically: look, but don't touch

@Richard From what you’ve written above, it sounds like EagleFiler handled the attachments properly. But feel free to contact me if you want to discuss further. It’s possible that Time Machine made an accurate backup but that not all of the attachments were cached locally, so they weren’t there to back up. You can avoid that problem in the future by having a rule that moves or copies the messages to an “On My Mac” mailbox or by periodically archiving the messages to EagleFiler.

So, I posted a note to apple communities asking 1. if Apple had acknowledged the bug, and 2. if not, how would I know when it is fixed?

I got this masterpiece of stupidity in response: Apple will neither acknowledge or discuss fixing a bug until it is fixed (assuming there is a bug.)


I've been fighting with this problem for a while and just did an interesting experiment. I sent myself two messages with attachments. One I downloaded to a local folder by dragging it to that folder. The message and attachment survived. The second I moved to the local folder via the Message - Move to drop-down menu. That message only retained its header (no contents or attachment). Has this been observed by others, and does it point to the problem?



@Mark I’m not sure what the pattern is. I have seen cases where all the different methods (drag and drop, menu, rule, AppleScript) worked and also cases where all of them damaged the message or made it disappear entirely.

OK. I would love to hear if what Doug did on Oct 21 fixed his issues. I updated to Catalina on Nov 30, by Dec 2, I was instructed by the mail client that my mailboxes had issues and before I could finish reading the little pop up window mail crashed. On relaunch mail rebuilt my mail boxes and deleted random emails. A lot of sent emails and all current emails between November 27 and Nov 30. The current ones were obvious, but the others only came to light when I was trying to recover some procedures I had emailed to someone back in May 2019. Those were just gone. After discovering that I started digging aound on the internet and on my machine. Discovered elmx files on timemachine existed, but completely gone on my computer. Called Apple and got no where fast. Spent that last two weekends rebuilding a machine with Mojave and migrating data between it and my current Catalina install... I have recovered from TimeMachine directly to the mojave install and that fails. I have restored the mail and my Library files to an external hard drive for safe keeping. Each time I recover my email from that hard drive and try to put it onto the mojave install the same emails disappear when Mail is relaunched. I am convinced it is because the mail server is an Exchange server, but the sent emails were sent from my computer mail client, not the server... it really doesn't make sense to me that the emails are just deleted off my hard drive by the mail client. I have the email in timemachine and on my external drive, but have been unsuccessful in getting them loaded back into Catalina or Mojave mail. Apple hasn't been able to offer a solution so I am curious if what Doug did with the manual import actually worked.

@Erin As far as I’m aware, the deletion at Mail launch is not related to Exchange. It also happens for locally stored messages.

@Erin -- from everything I've been able to show Apple, a manual "File --> Import Mailboxes" from a backup should be fine.

[…] We’re still hesitant to recommend upgrading to Catalina, particularly if you use Mail, since reports of email data loss continue. Luckily, downgrading a new Mac from Catalina to 10.14 Mojave is possible for models other than the […]

I had a huge mail archive. i normally use POP3 (53Gb). After the upgrade (yesterday) i had visualisation (and search) issue. Rebuilding the mailbox.... 6Gb. Lost 47Gb of mail. I now trying to recovery from back-up.

@Massimo -- please file a bug with Apple about this -- we need more bug reports to get action!

Something good just happened, but I have no idea why or if happened for others. A massive number (all?) of my formerly blank emails I had downloaded to my computer from my campus server now have their attachments back! The other good news, which makes me think this state of affairs may last, is that when I delete a message from the server using my computer it also deletes from what my iPhone sees on the server. I wonder if there may have been a fix at the Exchange side of thing. Anyway, I'm happy for now and will (cautiously) downloads the many messages I've been leaving on the server for safekeeping.



Sherman Wilcox

This is stunningly awful. I haven't filed a bug report because I never actually had a problem -- I abandoned Mail when I first heard reports. I'm now living quite happily in MailMate.

[…] Si eres de esos que aún no ha sido lobotomizados lo suficiente para abandonar el uso del correo electrónico, en macOS Catalina 💩 encontrarás un sinfín de problemas relacionados con […]

Where was this obvious issue before last Wed. Went from 15.1 to 15.2 and all of my saved folders with emails for various reasons. On top of this the conversion, it was totally screwed up the interfaces with the provider.

Went to apple support, the first two efforts were absolutely worthless [ very un apple support]. Hours spent bouncing between Provider and Apple. Apple wanted a reboot on Time Machine. That was and still is a mixed bag of bad issues. I will never let anyone lower than tier 2 to go though a time machine reboot.

Finally got a senior adviser. After 45 mins he said this needed engineering. He ran a number of screen shots and zip programs. we short of got thing working only going forward. It is still not stable. My guess we will have to back before 15.2 and go from to square one.

What really bothers me the most, Apple knows there is an issue and refuses to acknowledge it. Then have me running all over the place, causing more issues.

Same problem here with Catalina 10.15.2.
Unaware of all these problem with Apple Mail, and because I stupidly erased my mails (pop) while trying to change Pref. Syst./Internet Accounts (that's already strange: deleting an account will delete all related pop messages in Mail…), I decided to used my last Time Machine Backup.
Not working.
Ok, nothing to fear, I restore completely my MacBook Pro 16" with the last backup. Everything is working fine, but Apple Mail: wants to import messages, ok, and that's done, all messages are blank, just headers (and the lthe corresponding folder in Preferences/Mail is shrunk to 5% of its initial size: 54 GB to 2.5 GB...).
Not Happy.

that's already strange: deleting an account will delete all related pop messages in Mail

Yeah, that is so bad UX and it's been that way in Mail forever. Wonder how many out there have been bitten by this? There are so many more reasons to delete an account that just wanting to delete every message stored. Maybe you switch e-mail provider and just want to keep everything as an archive.

I can't talk about any public beta two of 10.15.3 to see if it fixes the issue with deleting the Envelope Index files/reimporting/reindexing "On My Mac" mail.

I would *like* to tell you to try that when it comes out to see if it fixes this problem on your end.

I would like to tell you it fixed my problems.

I really would.

All I can tell you -- it's a good day today. That I'm sure about.

@Shriner, can you not tell us whether or not this fix in the 10.15.3 beta simply repairs the index problem, or does it actually restore the lost e-mails?

I presume that it is not the latter, since, as others have noted, there is more free disk space after the Catalina upgrade, suggesting the actual deletion of e-mails and attachments, and not just an indexing issue.

I lost thousands of e-mails, with Gbs of additional free space after the upgrade.

I am anticipating having to go back to a TM backup from just prior to the Catalina upgrade, and importing e-mails, folder by folder, into the plethora of folder trees that I have, and then dealing with the duplication of e-mails between those imported, and the subset of e-mails that have, so far, survived the upgrade.

BTW, I did file *two* bug reports with Apple on this, since October.

Thank you for not telling us about this...

@Marc -- it will not restore e-mails if went through a process (for whatever reason) of reindexing your existing "On My Mac" mail. Those messages are gone for good and would need to be restored from a backup if that happened.

If you have a backup, you should be able to find the backed-up "On My Mac" folder in ~/Library/Mail/V6 and restore that and then "import mailboxes" successfully.

@Shriner, thanks for the clarification/confirmation. It is as I expected, given the observations of increased free disk space post upgrade.

All of the e-mails that I lost were in On My Mac folders. I generally do not keep e-mails in the cloud, with the exception of a few folders for portable and timely access on my iPhone when away from my MBP.

As a second question, I have noted some AppleScript tools online that purport to identify and delete duplicate e-mails. For example:


Any recommendations for such a tool based upon any experience?

I would guess that such a tool will be helpful to others in similar situations.

I will await the release of the 10.15.3 stable update and see what Apple has to finally say about this, before proceeding. This process of importing from TM and removing duplicate e-mails will be tedious and time consuming, thus only want to do it once. :-)


@Shriner, thanks for the weather report, very happy to hear about it being a good day as I'm about to have to commit to Catalina, and I just couldn't pull the trigger on moving to another client yet.

Can you also (not) tell us about any (alleged) resolution of the problem with moving messages between mailboxes? Is it a good day today as well?

@dpz -- I only personally move messages from my Gmail account to "on my mac" mailbox locations (that one way -- not the reverse).

So I'm not sure what that problem is directly nor what I'd be looking for to test against.

I would recommend watching for the public beta of 10.15.3 b2 and testing it?

@Marc -- no experience with "duplicate e-mail detectors" here, either. But, what I'd probably say would conceptually work would be that if you know the date you upgraded to Catalina, you could restore your "on my mac" mail from right before that upgrade, import it (which still works fine), and then just purge mail from the leftover "On My Mac" mailboxes from that date and before? That's what I did here. No duplicates remain doing that.

I am seeing some Exchange-related misbehaviour: all of a sudden, about 48 hours, my Catalina iMac stopped being able to fetch or synchronise email with my institute's Exchange server. New messages appear if I rebuild the mail boxes.

It was working absolutely fine before.

This occurs on both a wired iMac at my office and my home MBP on broadband. iOS mail and the Outlook web client work fine.

Moving lots of email out of the Inbox (from about 10k down to 1.7k) doesn't help.

Other accounts (Gmail, icloud) work fine on all machines.

This does not appear to be Catalina specific, however. Refs:

Is it, nonetheless, related? Any ideas?

Hi, I just tested the actual Dev-Beta (10.15.3 beta 2) with Mail 13.0 (3608. and I can no longer reproduce the mail-loss after rebuilding the envelope-index.
Can someone else confirm this?

I guess i was both sad and happy to find this article. I removed my envelope index files and one of my inbox lost about 110,000 emails. I have a super duper backup where the emails are intact which i can boot to, however i cant export the mailbox from apple mail it just crashes.

Can i replace my broken mail setup with the version from the super duper backup. However i have looked and i cant figure out how to do this. Can i just copy the V7 folder in its entirety ?

Secondly i did actually keep my envelope files before the issue. Can i restore them to fix the it ?

Also like another user here i actually deleted all my pop3 emails from my server as a way to debug the issue :(

Any help would be great. Alex

@Mathew -- yes, it seems fixed here as well.

@Alex -- what you should be able to do from your backup would be to find the "On My Mac" mail folders on your backup drive and "File --> Import Mailboxes" from Mail. The act of *manually importing them* -- is fine. Just don't delete the Envelope Index files as a method to "reimport" them until 10.15.3 is out

Apple fixing it or not, having lost and hence having had to restore from backup 200k mails, I no longer trust Apple worth my mails. I’ve moved away from Apple Mail. My company uses outlook anyway. I’ve set up DEVONthink Pro so that it imports my mails from Outlook. All my mail act on rules I’ve implemented in DEVONthink and I’ve linked them all together with a simple Apple script that I call from my DEVONthink toolbar. Effectively, I’ve now a very simple Auto File procedure that takes my Mails out of outlook (or Apple Mail, but as I said, I’m not using it anymore) and into my folder structure in DEVONthink.

Search and retrieval is anyway far superior in DEVONthink, so there’s no point going back even if Apple finally fixed it. They never really acknowledged the problem in the first place, hence why should I trust them.

Stay up-to-date by subscribing to the Comments RSS Feed for this post.

Leave a Comment