Apple Mail and Spotlight
John Gruber reports that Apple Mail in Tiger stores each POP message in its own file (à la Claris Emailer 1.0), rather than using one mbox file per mailbox. In previous versions of Mail, messages in mbox files were searchable using SearchKit, but in Tiger the messages must be available to Spotlight—and Spotlight uses files as the unit of search. Brent Simmons wonders whether other developers will change how their applications store data, in order to make it available to Spotlight. I think that would be a shame. If Mail does in fact shift to using one file per message, it will no longer be useful to me.
8 Comments RSS · Twitter
I agree, it would be a shame. I'm hoping that enough people will give feedback to apple on this and they will change how spotlight indexes.
Call me slow, really, but why does the switch to 1 file per message cause you pain? I read the article you wrote, which basically is a how-to on archiving your email. If Mail 2.0 works like a normal Maildir-based MUA, then you should be able to archive old messages more easily than an mbox-based MUA, because you should be able to move the email files themselves, at will.
You claim that using file-per-message will slow down your file system. Depending on your choice of file system, that may indeed be true. But, have you tested this under Tiger? I remember reading a file system comparison that showed that ReiserFS3, for instance, was faster when using small files than it was with large files. I grossly oversimplify, but...
You next say that it is not convenient to browse your mail when it is stored as file-per-message. I agree, it is indeed not convenient to browse your mail files, if you are using the file system, or a text editor. However, using a Maildir-enabled MUA, the problem goes away. And, assuming that Mail 2.0 uses a text file, and also assuming that it stores the message in RFC2822 (IIRC) compliant format, then that shouldn't be a problem.
You next say that it is easy to lose track of attachments. Again, assuming Mail 2.0 uses standard maildir, the attachment is encoded within each message, just as it is in the mbox format currently used, the only difference being that the messages are stored separately. How is this a problem?
Currently, I use Mail for both an IMAP and a POP account. In the POP account, the attachment is encoded in the message, in the IMAP account, it is stored on the server. If I choose to download the attachment, I am faced then with the choice of where to store it, and I wind up having to remember later which file has what purpose. How would that be any different from Mail 2.0?
Your final argument, that the tools for searching text are not currently optimized for searching email, is rather the point of Spotlight, I would have thought. Does Spotlight give you the ability to perform file system operations on the files you've found? If so, then you shouldn't have too much of a problem dealing with your email files. You would be able to move your email files somewhere useful, tar-and-zip them, burn the archive to CD, etc., etc., once you've used Spotlight to find the relevant files.
It seems to me that your reaction to the Maildir format news is premature. I'm sure we'll run into a bunch of surprises, and I'm also sure we'll run into a bunch of bugs, but I believe that your objections to Mail 2.0 are either not as bad as you think they are, or they will be handled gracefully in some way.
Of course, that's an awful lot of assumptions and suppositions on my part, too... :)
I've used e-mail programs that use one file per message with HFS+, and it's just not a pleasant experience when there's a lot of mail. The minimum file size overhead wastes disk space. I do not want to slow down backups and synchronizations by adding more than a million files to my home directory. And I think, but may be wrong, that this will make Mail slower. It made me happy when Apple started shipping its OS X Java VM with a few .jar files instead of thousands of .class files. :-)
Obviously, if Mail 2.0 uses a file per message, then it will become convienent to browse and search mail in that format. The attachment problems are specific to the programs mentioned in the article, and they don't apply when using Maildir format.
Well, hopefully it will all work out, and Apple will manage to tweak Tiger's performance in the right directions.
And if not, well, there's always, um, Mutt or something...
;)
I have 21,000+ mails in Tiger mail right now, with no slowdown whatsoever. It works just fine for me..
What I'm looking for is a separate application which will archive apple MAIL into dated folders. Is this redundant? Is it better simply to leave all my email in the mbox folders in apple mail and use spotlight? What if I want to have the mail available in text or rtf format for other applications (and not do it one by one using Save As...)? Any suggestions would be welcomed: mjscox@gmail.com
