{"id":43309,"date":"2024-05-17T16:19:04","date_gmt":"2024-05-17T20:19:04","guid":{"rendered":"https:\/\/mjtsai.com\/blog\/?p=43309"},"modified":"2024-05-22T14:21:27","modified_gmt":"2024-05-22T18:21:27","slug":"spamsieve-3-0-4","status":"publish","type":"post","link":"https:\/\/mjtsai.com\/blog\/2024\/05\/17\/spamsieve-3-0-4\/","title":{"rendered":"SpamSieve 3.0.4"},"content":{"rendered":"<p><a href=\"https:\/\/c-command.com\/blog\/2024\/05\/17\/spamsieve-3-0-4\/\">SpamSieve 3.0.4<\/a> is a maintenance release for my Mac e-mail spam filter.<\/p>\n\n<p>In sending out the update notice to the <a href=\"https:\/\/c-command.com\/spamsieve\/support#news\">SpamSieve mailing list<\/a>, Amazon SES reported a huge number of bounces (almost 10x the normal percentage), primarily from EarthLink\/MindSpring addresses. It&rsquo;s not clear what&rsquo;s going on here, but I suspect that the addresses are not all suddenly bad. Unfortunately, Sendy seems to want to permanently disable them.<\/p>\n\n<p>Some interesting bugs were:<\/p>\n<ul>\n<li><p>Sometimes SpamSieve would send an AppleScript command to Apple Mail but never receive a response. Sampling the process would show that Mail was not busy formulating a reply; it simply seemed to drop the request. I&rsquo;m not exactly sure what causes this&mdash;perhaps Mail being overloaded&mdash;but I&rsquo;ve made lots of changes so that SpamSieve can better handle this situation. We don&rsquo;t want to just repeatedly wait for a long timeout.<\/p><\/li>\n<li><p>There was a related issue, worked around in a previous version, where Mail could take a really long time to read or download a message, and SpamSieve would try to detect this and throttle requests. However, the workaround had a bug where, if the user put the Mac to sleep in the middle of a request, that would mess up SpamSieve&rsquo;s estimate of the throughput.<\/p><\/li>\n<li><p>A damaged database may open and fetch without error but then fail to save. SpamSieve tries to handle save errors by either recovering or (as in the case of corruption or a disk error) rolling back. The most important thing is to avoid growing an unbounded graph of objects in memory. This didn&rsquo;t always work properly because I had failed to account for the fact that Core Data errors may have <code>NSSQLiteErrorDomain<\/code>. I had expected that such errors would only occur as underlying errors inside of a <code>CocoaError<\/code>.<\/p><\/li>\n<li><p>Some Apple Mail operations could be unnecessarily slow or even crash because my AppleScript had a typo inside of a <code>try\/end<\/code> block. The intent was to just suppress any error (since it was in debug code). Unfortunately, the typo meant that an error was generated <em>every<\/em> time through this code path. The error was related to referencing a selected message in Mail, so if many messages were selected, AppleScript would generate an error message that included a giant string like <code>item 1 of {long list of references to messages, each with the mailbox and account name}<\/code> even though this was never caught or displayed. This would take a long time and, I think, potentially cause a crash.<\/p><\/li>\n<li><p><code>_PFBatchFaultingArray<\/code> is a leaky abstraction that will raise an exception (not catchable in Swift) if it&rsquo;s accessed after the persistent store is shut down. I think I fixed the order of cleanup to prevent this and also made sure that all accesses to such arrays go through an Objective-C category that catches exceptions.<\/p><\/li>\n<li><p>I wasn&rsquo;t able to fully isolate the problem, but <code>NSRegularExpression<\/code> sometimes raises an exception (not catchable in Swift) about an invalid range when using a range created by <code>NSRange(swiftRange, in: swiftString)<\/code>. It works better to to avoid Swift ranges entirely and use <code>NSRange<\/code> and the associated <code>NSString<\/code> methods when using Swift strings with <code>NSRegularExpression<\/code>.<\/p><\/li>\n<\/ul>\n\n<p>Previously:<\/p>\n<ul>\n<li><a href=\"https:\/\/mjtsai.com\/blog\/2023\/12\/27\/spamsieve-3-0-3\/\">SpamSieve 3.0.3<\/a><\/li>\n<\/ul>\n\n<p id=\"spamsieve-3-0-4-update-2024-05-22\">Update (2024-05-22): I found that the bounced addresses were mostly ones that were added a long time ago, and after sending test messages to a sample of them I got no replies and lots of  bounces. So I think they really are invalid, though it remains a mystery why they were all reported as such now, rather than with previous messages. Thanks to Ben at Sendy for for explaining that it&rsquo;s possible to remove the bounce status in Sendy by deleting the addresses and reimporting them. It&rsquo;s also possible to use Amazon SES&rsquo;s <a href=\"https:\/\/docs.aws.amazon.com\/ses\/latest\/dg\/sending-email-suppression-list.html\">suppression list<\/a> to tell it to try certain addresses again, though this can be risky for your account if they really are bad. SES&rsquo;s SMTP server is, of course, no good for manual testing because it will silently skip sending the message to addresses that are already on its global suppression list.<\/p>","protected":false},"excerpt":{"rendered":"<p>SpamSieve 3.0.4 is a maintenance release for my Mac e-mail spam filter. In sending out the update notice to the SpamSieve mailing list, Amazon SES reported a huge number of bounces (almost 10x the normal percentage), primarily from EarthLink\/MindSpring addresses. It&rsquo;s not clear what&rsquo;s going on here, but I suspect that the addresses are not [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"apple_news_api_created_at":"2024-05-17T20:19:07Z","apple_news_api_id":"ca911e73-c480-44d8-8908-b124a85dd0b0","apple_news_api_modified_at":"2024-05-22T18:21:31Z","apple_news_api_revision":"AAAAAAAAAAAAAAAAAAAABQ==","apple_news_api_share_url":"https:\/\/apple.news\/AypEec8SARNiJCLEkqF3QsA","apple_news_coverimage":0,"apple_news_coverimage_caption":"","apple_news_is_hidden":false,"apple_news_is_paid":false,"apple_news_is_preview":false,"apple_news_is_sponsored":false,"apple_news_maturity_rating":"","apple_news_metadata":"\"\"","apple_news_pullquote":"","apple_news_pullquote_position":"","apple_news_slug":"","apple_news_sections":"\"\"","apple_news_suppress_video_url":false,"apple_news_use_image_component":false,"footnotes":""},"categories":[4],"tags":[602,126,159,69,109,143,30,32,2385,1172,71,1010,2062,372,425,901],"class_list":["post-43309","post","type-post","status-publish","format-standard","hentry","category-programming-category","tag-amazon-ses","tag-applemail","tag-applescript","tag-cocoa","tag-coredata","tag-database","tag-mac","tag-macapp","tag-macos-14-sonoma","tag-mailing-lists","tag-programming","tag-sendy","tag-sleep-mode","tag-spamsieve","tag-sqlite","tag-swift-programming-language"],"apple_news_notices":[],"_links":{"self":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/43309","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/comments?post=43309"}],"version-history":[{"count":7,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/43309\/revisions"}],"predecessor-version":[{"id":43377,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/43309\/revisions\/43377"}],"wp:attachment":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/media?parent=43309"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/categories?post=43309"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/tags?post=43309"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}