{"id":36821,"date":"2022-08-22T16:52:45","date_gmt":"2022-08-22T20:52:45","guid":{"rendered":"https:\/\/mjtsai.com\/blog\/?p=36821"},"modified":"2022-10-10T10:57:01","modified_gmt":"2022-10-10T14:57:01","slug":"google-account-deleted-due-to-csam-false-positive","status":"publish","type":"post","link":"https:\/\/mjtsai.com\/blog\/2022\/08\/22\/google-account-deleted-due-to-csam-false-positive\/","title":{"rendered":"Google Account Deleted Due to CSAM False Positive"},"content":{"rendered":"<p><a href=\"https:\/\/www.nytimes.com\/2022\/08\/21\/technology\/google-surveillance-toddler-photo.html\">Kashmir Hill<\/a> (<a href=\"https:\/\/news.ycombinator.com\/item?id=32538805\">Hacker News<\/a>):<\/p>\n<blockquote cite=\"https:\/\/www.nytimes.com\/2022\/08\/21\/technology\/google-surveillance-toddler-photo.html\">\n<p>With help from the photos, the doctor diagnosed the issue and prescribed antibiotics, which quickly cleared it up. But the episode left Mark with a much larger problem, one that would cost him more than a decade of contacts, emails and photos, and make him the target of a police investigation. Mark, who asked to be identified only by his first name for fear of potential reputational harm, had been caught in an algorithmic net designed to snare people exchanging child sexual abuse material.<\/p>\n<p>[&#8230;]<\/p>\n<p>Two days after taking the photos of his son, Mark&rsquo;s phone made a blooping notification noise: His account had been disabled because of &ldquo;harmful content&rdquo; that was &ldquo;a severe violation of Google&rsquo;s policies and might be illegal.&rdquo;<\/p>\n<p>[&#8230;]<\/p>\n<p>A few days after Mark filed the appeal, Google responded that it would not reinstate the account, with no further explanation.<\/p>\n<p>[&#8230;]<\/p>\n<p>CyberTipline staff members add any new abusive images to the hashed database that is shared with technology companies for scanning purposes. When Mark&rsquo;s wife learned this, she deleted the photos Mark had taken of their son from her iPhone, for fear Apple might flag her account.<\/p>\n<\/blockquote>\n\n<p>The police determined that no crime had occurred, but Google permanently deleted his account, anyway. Apparently, the police now have the only copy of his data.<\/p>\n\n<p>I don&rsquo;t really want to use iCloud Photo Library, but I have it enabled now because Image Capture doesn&rsquo;t work wirelessly, and recent versions have been buggy. I guess the proper way to take photos for a doctor would be to temporarily turn off iCloud Photo Library or to use a third-party camera app that doesn&rsquo;t save to the camera roll. But I bet nearly every iPhone user has some photos&mdash;be they medical, sexual, or of documents&mdash;that they would like to mark as private (not just <a href=\"https:\/\/appletoolbox.com\/how-to-set-photos-as-private-on-iphone\/\">hidden<\/a>). They should still be backed up but protected with an extra password or something. I don&rsquo;t know how to prevent this from being abused to store actual CSAM, though.<\/p>\n\n<p><a href=\"https:\/\/twitter.com\/meekgeek\/status\/1561580430641741824\">Meek Geek<\/a>:<\/p>\n<blockquote cite=\"https:\/\/twitter.com\/meekgeek\/status\/1561580430641741824\">\n<p>If you are accused by Google of doing something they don&rsquo;t like and have your account blocked, there is no easy way to get human support on the other side to review the issue.<\/p>\n<\/blockquote>\n\n<p><a href=\"https:\/\/twitter.com\/Freerunnering\/status\/1561725130933420033\">Kyle Howells<\/a>:<\/p>\n<blockquote cite=\"https:\/\/twitter.com\/Freerunnering\/status\/1561725130933420033\">\n<p>One of the things I&rsquo;ve been doing the last few years is trying to slowly remove Google as a single point of failure in my life.<\/p>\n<p>Spreading out my online life over more companies so no 1 company can ruin my life at the flick of a switch.<\/p>\n<\/blockquote>\n\n<p>There&rsquo;s no real way to remove Apple if you use an iPhone.<\/p>\n\n<p>Previously:<\/p>\n<ul>\n<li><a href=\"https:\/\/mjtsai.com\/blog\/2022\/08\/19\/samsungs-repair-mode\/\">Samsung&rsquo;s Repair Mode<\/a><\/li>\n<li><a href=\"https:\/\/mjtsai.com\/blog\/2022\/08\/17\/smartphones-a-single-point-of-failure\/\">Smartphones: a Single Point of Failure<\/a><\/li>\n<li><a href=\"https:\/\/mjtsai.com\/blog\/2021\/12\/07\/apple-accounts-permanently-blocked\/\">Apple Accounts &ldquo;Permanently&rdquo; Blocked<\/a><\/li>\n<li><a href=\"https:\/\/mjtsai.com\/blog\/2021\/10\/15\/the-risks-of-client-side-scanning\/\">The Risks of Client-Side Scanning<\/a><\/li>\n<li><a href=\"https:\/\/mjtsai.com\/blog\/2021\/09\/07\/apple-delays-child-safety-features\/\">Apple Delays Child Safety Features<\/a><\/li>\n<li><a href=\"https:\/\/mjtsai.com\/blog\/2021\/08\/31\/privacy-for-apple-employees\/\">Privacy for Apple Employees<\/a><\/li>\n<li><a href=\"https:\/\/mjtsai.com\/blog\/2021\/08\/05\/scanning-icloud-photos-for-child-sexual-abuse\/\">Scanning iCloud Photos for Child Sexual Abuse<\/a><\/li>\n<li><a href=\"https:\/\/mjtsai.com\/blog\/2021\/06\/11\/settlement-for-applecare-privacy-invasion\/\">Settlement for AppleCare Privacy Invasion<\/a><\/li>\n<li><a href=\"https:\/\/mjtsai.com\/blog\/2021\/03\/04\/apple-account-locked-due-to-failed-trade-in\/\">Apple Account Locked Due to Failed Trade-in<\/a><\/li>\n<li><a href=\"https:\/\/mjtsai.com\/blog\/2020\/10\/06\/stolen-instagram-account\/\">Stolen Instagram Account<\/a><\/li>\n<li><a href=\"https:\/\/mjtsai.com\/blog\/2014\/09\/01\/apple-patches-find-my-iphone-exploit\/\">Apple Patches &ldquo;Find My iPhone&rdquo; Exploit<\/a><\/li>\n<\/ul>\n\n<p id=\"google-account-deleted-due-to-csam-false-positive-update-2022-08-26\">Update (2022-08-26): <a href=\"https:\/\/daringfireball.net\/linked\/2022\/08\/22\/hill-csam\">John Gruber<\/a>:<\/p>\n<blockquote cite=\"https:\/\/daringfireball.net\/linked\/2022\/08\/22\/hill-csam\">\n<p>To my knowledge, no innocent person has been falsely flagged and investigated like Mark using the NCMEC fingerprint database. It could happen. But I don&rsquo;t think it has. It seems uncommon for an innocent person like Mark to be flagged and investigated by the second method, but as Hill reports, we have no way of knowing how many like Mark there are who&rsquo;ve been wrongly flagged, because for obvious reasons they&rsquo;re unlikely to go public with their stories.<\/p>\n<p>[&#8230;]<\/p>\n<p>&ldquo;Avoid uploading to the cloud&rdquo; is difficult advice for most people to follow. Just about everyone uses their phone as their camera, and most phones from the last decade or so&#x2009;&mdash;&#x2009;iPhones and Android alike&#x2009;&mdash;&#x2009;upload photos to the cloud automatically. When on Wi-Fi&#x2009;&mdash;&#x2009;like almost everyone is at home&#x2009;&mdash;&#x2009;the uploads to the cloud are often nearly instantaneous.<\/p>\n<p>[&#8230;]<\/p>\n<p>The on-device vs. on-server debate is legitimate and worth having. But I think it ought to be far less controversial than Google&rsquo;s already-in-place system of trying to identify CSAM that isn&rsquo;t in the NCMEC known database.<\/p>\n<\/blockquote>\n\n<p>See also: <a href=\"https:\/\/twitter.com\/ditheringfm\/status\/1562017115690872833\">Dithering<\/a>, <a href=\"https:\/\/daringfireball.net\/thetalkshow\/2022\/08\/26\/ep-355\">The Talk Show<\/a>.<\/p>\n\n<p id=\"google-account-deleted-due-to-csam-false-positive-update-2022-10-10\">Update (2022-10-10): See also: <a href=\"https:\/\/news.ycombinator.com\/item?id=32560361\">Hacker News<\/a>, <a href=\"https:\/\/stratechery.com\/2022\/rights-laws-and-google\/\">Ben Thompson<\/a>, <a href=\"https:\/\/pxlnv.com\/blog\/free-as-in-context\/\">Nick Heer<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"<p>Kashmir Hill (Hacker News): With help from the photos, the doctor diagnosed the issue and prescribed antibiotics, which quickly cleared it up. But the episode left Mark with a much larger problem, one that would cost him more than a decade of contacts, emails and photos, and make him the target of a police investigation. [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"apple_news_api_created_at":"2022-08-22T20:52:48Z","apple_news_api_id":"aca83feb-f618-405e-af37-f558edb85047","apple_news_api_modified_at":"2022-10-10T14:57:04Z","apple_news_api_revision":"AAAAAAAAAAAAAAAAAAAAAg==","apple_news_api_share_url":"https:\/\/apple.news\/ArKg_6_YYQF6vN_VY7bhQRw","apple_news_coverimage":0,"apple_news_coverimage_caption":"","apple_news_is_hidden":false,"apple_news_is_paid":false,"apple_news_is_preview":false,"apple_news_is_sponsored":false,"apple_news_maturity_rating":"","apple_news_metadata":"\"\"","apple_news_pullquote":"","apple_news_pullquote_position":"","apple_news_slug":"","apple_news_sections":"\"\"","apple_news_suppress_video_url":false,"apple_news_use_image_component":false,"footnotes":""},"categories":[2],"tags":[1351,2106,1016,433,51,1208,1142,1268,31,2078,355],"class_list":["post-36821","post","type-post","status-publish","format-standard","hentry","category-technology","tag-artificial-intelligence","tag-child-sexual-abuse-material-csam","tag-datacide","tag-gmail","tag-google","tag-google-photos","tag-icloud-photo-library","tag-image-capture","tag-ios","tag-ios-15","tag-privacy"],"apple_news_notices":[],"_links":{"self":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/36821","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/comments?post=36821"}],"version-history":[{"count":4,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/36821\/revisions"}],"predecessor-version":[{"id":37280,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/36821\/revisions\/37280"}],"wp:attachment":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/media?parent=36821"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/categories?post=36821"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/tags?post=36821"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}