SpamSieve 3.0
I’ve been away from blogging for a few days because I’ve been busy finishing up SpamSieve 3, getting it released, and responding to the onslaught of support e-mails. This is a major upgrade, which rewrites most of the app, both at the user interface level and internally. (Hello to Swift, Core Data, and a new e-mail parser and localization system; goodbye to nibs, custom file formats, and many lines of Objective-C. Some things like using lots of AppKit and AppleScript are constants.) This all enables more accurate spam filtering, makes the app easier to set up and use, and enables lots of new features—some available now and others planned for the future.
Perhaps I’ll write about some of that later. For now, the most important point is that SpamSieve 2 won’t work with Apple Mail on Sonoma—which is arriving on Tuesday—because Sonoma removes support for Mail plug-ins. And SpamSieve 3 will work on Sonoma because the Mail integration has been rewritten using a Mail extension (plus lots of AppleScript). Besides updating the SpamSieve app, there are some other changes you’ll want to understand. I’ve written a guide explaining the transition.
So far, the update has gone very well at an engineering level. I was nervous about changing so many parts of the app at once, but thanks to careful design, lots of internal unit testing and stress testing, and a great group of beta testers, no significant bugs have been reported yet. Of course, there are a bunch of small issues that will be cleaned up and polished in 3.0.1 and beyond, but I’m really happy with the state of the app. I think Kenichi Yoshida did great job with the new app icon and the status indicators within it.
I also want to thank all of the localizers. Version 3 is launching with support for German, Spanish, French, Japanese, and Dutch. Swedish is in the works. (Let me know if you would like to help add another language.) The localizers and international resellers (ASH and Infinisys) did a lot of work on short notice to get everything ready before Sonoma was released. There have been a few minor issues with the Paddle and FastSpring stores, but for the most part they’ve worked very well.
What has not gone so well is the mailing lists for contacting my own customers. Here I changed nothing from what’s worked very well in the past, yet it seems as though many—maybe even most—customers did not receive the e-mails announcing the new version or highlighting how to handle the update to Sonoma. Even I didn’t receive the latter. It took me a while to realize this because I was getting tons of e-mails, but they were almost all individually written, not replies to the list messages, like normal. I’m in communication with DreamHost to see if they can figure out what happened and how many e-mails were actually delivered, but at this point I think I need to find an alternate solution. I’m looking into setting up Sendy. Yes, it is ironic that I’m running into trouble sending—rather than stopping—mass e-mails. And also that I’m going to send some people duplicate e-mails because it’s important enough that everyone be apprised of the situation with Sonoma.
Previously:
- macOS 14 Sonoma Shipping Soon
- RIP Apple Mail Plug-ins
- Mail App Extensions
- Working Without a Nib
- Whoosh 2.0
20 Comments RSS · Twitter · Mastodon
Michael, you were a beast getting betas out and keeping moving towards the release, You had issues fixed before I could get a chance to report them. Congratulations of the release 3.0, and looking forward to 3.0.1 when you’ve had a chance to rest. Thanks for all that you do to keep my mailbox spamless.
Congrats Michael, on the big new release!
If you're still looking for email sending / marketing solutions I can recommend Direct Mail for the Mac. It's a native Mac app with a lot of great features, easy to use and reasonably priced (with a pay per email option). Email sending can occur through your own SMTP server or their server. Have also found their support excellent.
@Matthew Thanks. Direct Mail sounds promising and perhaps easier to set up, but their Web site seems to be down. It’s also not clear to me how people would subscribe/unsubscribe if it’s not server-based? The App Store description doesn’t mention that.
@Michael - the website is https://directmailmac.com and seems to be working now.
The architecture is a hybrid client-server model so mailing lists etc can be stored on the Direct Mail server and they provide sign-up form code which submits to this server. The Mac app then acts as the client and UI.
Congrats on the release!
I use Sendy and am generally very happy (having previously used campaign monitor, which has similar features but is 100x as expensive). SES is the mail sending service we use with it.
You really shouldn't send emails from your domain. I looked a couple of times at Sendy but I don't want to install or manage anything of that size on my server.
There is always a deal with a good price at AppSumo for a decent email sending/marketing company.
Although I do understand your rationale for continuing to integrate tightly with clients (thanks for Sonoma Mail support whilst maintaining the plug-in for older releases, just upgraded now) I'm sorry you didn't take the opportunity to include direct support for IMAP. You would then be in a very good position to support completely automated/drone setups and also provide added value e.g. general-purpose filtering. Maybe version 4?
As for volume sending, the important part is the output stage i.e. the server that your application uses to reach receivers. You'll probably just end up doing what everyone else does and just use a service like Amazon SES (cheap, fast, reputable) but nothing stops you doing it yourself if you really want to. Sendy has direct support for SES's special features via their API, but other options are available with SMTP interfaces that you tell your mailer to use (like a separately configured list manager or mail server, the latter great for reducing latency of each send). Details of how you do this will be offered by each service. The one constant in my opinion is that getting your hosting provider to do this, or a free provider, is NOT a great idea—these are really meant for a general mix of general mail, rather than efficient and well-tracked mailing list deliveries. Actually using your domain should not be a problem though, if your authentication records are set up properly. Good luck.
Congrats on the awesome release!
I've seen you've made it to Germany's biggest tech-news-site: https://www.heise.de/news/SpamSieve-3-0-Komplettumbau-wegen-Sonoma-9315230.html
@Sebby I think direct client integration gives a better user experience in a variety of ways, plus it works with Exchange and POP, too. But, yes, direct IMAP support is something I’m considering for a future version—this was already too much stuff in one release—as it would have a separate set of benefits.
I decided to go with Sendy and hooked it up to my existing Amazon SES account. It seems to work very well.
@Karsten Thanks!
Update: Sendy seems to not be able to handle SES pausing your account.
Michael- I recently upgraded to the latest version of SpamSieve. I have been a user for years and years. After installing I tried to cleaning up some folders and did a mass delete on a junk box. Whatever was in there caused SpamSieve to reject some of these (because of two accounts with the same name) - which would be fine, except it is causing a pop-up that needs to have the "cancel" button hit for every one of those deleted messages. This is hundreds. Is there any way to stop SpamSieve from staying in this loop?? -Jeff
@Jeff Please contact spamsieve@c-command.com for support questions. But based on what you’ve written here, this doesn’t sound related to SpamSieve. It is not involved in message deletion.
I just got your mailing list announcement for the first time, so I'm among those who didn't get them previously. I presume your new arrangements for email went awry. What happened, and which software are you on now, out of curiosity?
@Sebby I’m using Sendy with SES and Mailgun. Partway through the first message sent with Sendy, Amazon paused my account because there were too many bounces (minutes after they e-mailed me to say it wasn’t going to be a problem). It turns out that for over a decade the DreamHost mailing list software had not been cleaning bounced addresses like it said it was, so there was too high a percentage of bad addresses for SES’s liking. It took a few days going back and forth with Sendy’s support to figure out which addresses failed after the SES shutdown and so needed to be retried. I guess you were in that batch. I then used a service to clean that portion of the address list and sent out the remaining e-mails via Mailgun (since I’m still arguing with Amazon to get SES reinstated).
Summary at the moment: SES is a terrific service with poorly written admin e-mails and slow, unhelpful support. Sendy is a good app with poor documentation and slow but OK support. Mailgun is slower and more expensive than SES (and doesn’t integrate as well with Sendy) but has fast, excellent support. DreamHost is overall a good host with good support, but don’t use their Mailman lists or bespoke announce lists. Glad that I had a backup provider configured already (including SPF) so that my transactional e-mails automatically switched over to Mailgun when SES started returning errors. Unfortunately, Sendy doesn’t know how to do that and doesn’t have built-in support for retries.
@Michael thanks for the update. Sigh though, if there's one thing I think SES really fails at it's bounces. Very, very aggressive, and their heuristics mean you really want to parse the JSON they send to SNS rather than trying to parse the forwarded bounces and use a relaxed scoring system typically used by other list software, because they e.g. treat SMTP hard policy rejections as transient equivalent to temporary 4xx errors, but "mailbox doesn't exist" as hanging offences for which not too many get you "reviewed". It's very unfortunate, but somehow inevitable, both that the Amazon logic has a crazy common sense to it because too many sysadmins hardfail for the silliest of reasons, but that software (like Sendy) would then come to depend on the Amazon heuristics, in effect locking you in to SES. Perhaps you should give thought to constructing and sending your own mails; then you can use whatever provider you want and be sure you instantly learn about any holdups or failures. I am glad that you have had a backup strategy in place that has worked for you. And I don't think you should hold yourself to an imaginary standard of probity; even the best-maintained lists now have delivery issues because of how aggressively the email infrastructure is policed. It's just the way things have (sadly) turned out.
@Sebby Yes, I don’t understand why SES is so strict about “mailbox doesn’t exist” errors. They don’t even care about specific mailboxes, just the percentages. They told me that, if I’m not sure an address is good, to send out more e-mails to known good addresses at the same time to make the bounce percentage lower.
Sendy does do special error handling for SES via SNS, and it does let you configure an SMTP server instead. But the only way to switch services in the middle of sending a campaign is to kill the server process after you change the setting. I was reluctant to do that before hearing back from support, at which point it was too late and SES was paused.
I was thinking about writing my own utility, but now that I understand how Sendy works better, I think it’s sufficient for my purposes.
@Michael Well, certainly having invalid addresses on your list isn't healthy; properly functioning bounce detection would be aware of the scale of messages sent over time and remove persistently non-working addresses. My guess is that Amazon chooses to aggressively screen "no such mailbox" because many of them are former addresses, which will soon reopen as spamtraps that would hurt Amazon's reputation. Again, there is a crazy rationale in this; the only people who need to hurt before things get serious for Amazon are Amazon's own customers. So they advise you to start slow, trickle out your mail at a rate that is unlikely to offend, and eventually you can "listwash" your list and pick up a rate that is satisfactory. I guess that now you've got through your list, you don't need to do this anymore, but I've seen people suggest address validation services be used prior to sending, which try to find invalid addresses by using "SMTP interruptus" without sending mail, before doing the initial (real) send. These won't catch everything, but would probably eliminate the bulk of the failing addresses which are immediately rejected at SMTP time. I still think Amazon's strategy, though it makes sense for maximal efficiency and minimum overall harm, is ultimately bad for senders, who need some initial period of grace to validate their lists before they can meet the promised targets for valid recipients by volume. Again, the enshittification of email.
As to Sendy, I guess the only thing I'd say is that if you run your own queueing MTA, then it can hold mail in the queue whilst you wait for your accounts to become available, or to fall back to backup relays; it also makes your life much easier for sending, since all the business of authentication, encryption, parallelism, etc. is managed by the MTA. OpenSMTPD is easily installed from MacPorts, simple to configure and can be told to relay to SMTP servers of choice, then your applications talk to your relay, which does all the retrying. Just something to consider.
