Facebook Solicits Nude Photos to Stop Revenge Porn
As part of a new feature the social network is testing in Australia, users are being asked to upload explicit photos of themselves before they send them to anyone else, according to the Australian Broadcasting Corporation (ABC).
[…]
The social network then builds what is referred to as a “hash” of the image, meaning it creates a unique fingerprint for the file. Facebook says it is not storing the photos, just the hashes of the photos. If another user tries to upload the same image on Facebook or Instagram, Facebook will test it against its stored hashes, and stop those labeled as revenge porn from being distributed.
Joseph Cox (tweet):
What that and other explanations do not necessarily make clear, however, is that prior to making that fingerprint, a worker from Facebook’s community operations team will actually look at the uncensored image itself to make sure it really is violating Facebook’s policies.
[…]
“It is absolutely necessary for images to be reviewed by a person when introduced into the checked for dataset, otherwise it would be trivial for someone to abuse this process to censor images,” Nicholas Weaver, a senior researcher at the International Computer Science Institute in Berkeley, California, told The Daily Beast. Weaver pointed at the iconic photo of Tank Man in Tiananmen Square as an example.
In a bizarre way, this actually makes some sense: Facebook already bans pornography, but there’s no algorithmic way to determine if a photo was shared non-consensually, so a user must manually state that certain images were shared without their consent. The distinction is important because someone sharing consensual porn is merely violating Facebook’s terms of use, while someone sharing non-consensual images is violating a person’s privacy and, potentially, the law.
I’m not sure I like this. It doesn’t prevent revenge porn in general; it only prevents the same photos being uploaded to Facebook in particular.
Previously: Photos Machine Learning and Trusting Apple.
Update (2017-11-09): Wil Shipley:
Facebook could have said: “Here’s a tool for you to create hashes of anything you’re afraid someone might post in revenge. Send us the hashes and if we see matching posts we’ll evaluate their content then and take them down if needed.”
It's all fun and games until someone uploads the Mona Lisa or pictures of the ocean and it starts flagging the wrong stuff.